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Abstract 

Decisions  in  which  multiple  objectives  must  be  optimized  simultaneously  occur  fre¬ 
quently  in  government,  military,  and  industrial  settings.  One  method  a  decision  maker 
(e.g.,  a  design  engineer)  may  use  to  assist  in  multiple  response  optimization  situations 
is  the  application  of  a  desirability  function.  The  decision  maker  specifies  the  desirabil¬ 
ity  function  parameters  so  as  to  express  his  or  her  own  preferences  with  respect  to  the 
objectives  under  consideration.  An  informed  specification  of  the  parameters  is  essen¬ 
tial  so  that  the  desirability  function  accurately  describes  the  decision  maker’s  value 
trade-offs  and  risk  preference.  Misapplication  of  the  desirability  function  may  result 
in  the  selection  of  an  optimal  policy  that  is  inconsistent  with  the  stated  preferences. 
This  thesis  examines  the  desirability  function  from  a  decision  analysis  perspective. 
In  particular,  utility  transversality  provides  the  basis  for  an  analysis  of  the  implicit 
value  trade-off  and  risk  attitude  assumptions  attendant  to  the  desirability  function. 

A  limitation  of  the  desirability  function  is  its  failure  to  explicitly  account  for 
response  variability.  A  robust  solution  accounts  for  not  only  the  expected  response, 
but  the  variance  as  well.  Assessing  a  utility  function  over  desirability  as  a  means  to 
describe  the  decision  maker’s  risk  preference  produces  a  robust  operating  solution  that 
is  consistent  with  those  preferences.  This  thesis  examines  robustness  as  it  applies  to 
the  desirability  function,  using  a  decision  analysis  perspective.  In  particular,  a  robust 
manufacturing  solution  is  identified  for  a  wire-bonding  process,  seen  often  in  the 
quality  and  reliability  engineering  design  literature.  An  exponential  utility  function 
over  desirability  is  applied  to  regression  equations  developed  from  a  Box-Behnken 
design.  Monte  Carlo  simulation  enables  specification  of  the  robust  solution. 
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Using  decision  analysis  methods,  this  methodology  is  applied  to  a  practical  prob¬ 
lem  currently  facing  the  Air  Force  Research  Laboratory  (AFRL).  Contributing  to 
AFRL’s  Robust  Decision  Making  Strategic  Technology  Team  program,  this  thesis 
examines  robustness  in  the  context  of  national  policy-making  in  country  stability  sit¬ 
uation.  Different  levels  of  diplomatic,  informational,  military,  and  economic  (DIME) 
instruments  of  national  policy  are  investigated  to  examine  how  they  affect  the  po¬ 
litical,  military,  economic,  social,  infrastructure,  and  information  (PMESII)  systems 
of  a  nation.  AFRL’s  National  Operational  Environment  Model  (NOEM)  serves  as 
the  basis  for  the  analysis  of  a  scenario  involving  the  Democratic  Republic  of  Congo. 
A  D-optimal  design  of  experiments  enables  identification  of  a  robust  national  policy. 
Employment  of  a  multiattribute  utility  function  that  satisfies  the  axioms  of  expected 
utility  theory  ensures  that  the  policy  is  consistent  with  the  decision  maker’s  stated 
preferences. 
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A  DECISION  ANALYSIS  PERSPECTIVE  ON  MULTIPLE  RESPONSE  ROBUST 

OPTIMIZATION 

I.  Introduction 

Often,  a  decision  must  be  made  that  optimizes  a  single  objective,  such  as  a  com¬ 
pany  desiring  to  maximize  profit.  Other  times,  decisions  must  be  made  to  simulta¬ 
neously  optimize  a  set  of  multiple  objectives.  Such  decisions  can  occur  within  any 
organization. 

Examples  of  multiple  objective  decisions  in  government  include  selecting  the  proper 
regional  road  network  that  balances  the  environmental,  social  and  economic  impacts 
of  the  region’s  municipalities  within  a  fixed  budget  [7] .  Another  example  is  selecting 
the  best  location  for  a  new  airport.  The  competing  objectives  in  this  case  are  minimiz¬ 
ing  construction  costs,  while  also  minimizing  transit  time  for  travelers,  maximizing 
safety  and  minimizing  noise  pollution  [21]. 

Examples  in  military  settings  include  designing  a  military  aircraft  where  maximum 
range  and  maximum  speed  are  two  competing  objectives  with  maximum  payload 
capacity  [25].  Another  example  is  developing  a  stabilization,  security,  transition, 
and  reconstruction  operation  (SSTRO)  where  the  nation’s  positive  political,  military, 
economic,  and  social  indicators  are  desired  [5]. 

Examples  in  industry  include  gum  extraction  from  plant  seeds  where  maximum 
extraction  yield,  viscosity,  hue,  and  emulsion  stability  needs  to  be  balanced  with 
minimum  protein  content  [28] .  Another  example  is  machining  parts  where  the  balance 
of  metal  removal  rate  and  surface  roughness  needs  to  be  optimized  [35] .  One’s  personal 
life  also  holds  examples  of  such  decisions  such  as  buying  a  truck  where  one  desires 
maximum  power  with  maximum  fuel  economy. 
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Often,  objectives  conflict  so  that  selecting  an  alternative  resulting  in  the  simul¬ 
taneous  optimum  for  each  objective  is  infeasible.  Consider  the  truck  buyer  desiring 
increased  power  in  a  new  truck  with  increased  fuel  economy.  Nearly  always,  the  truck 
with  the  most  power  will  not  also  have  the  highest  fuel  economy.  This  “ideal”  truck 
does  not  exist.  In  these  cases,  a  compromise  solution  to  balance  these  competing  ob¬ 
jectives  must  be  found.  This  correct  balance  is  based  in  large  part  on  the  application 
and  objectives  in  question  as  well  as  the  preferences  of  the  decision  makers  in  charge. 
Although  many  organizations  have  a  group  of  people  making  these  types  of  decisions, 
this  thesis  assumes  a  single  decision  maker  with  no  loss  of  generality.  Several  areas 
of  study  exist  to  assist  the  decision  maker  with  such  decisions. 

Decision  analysis  is  a  combination  of  mathematical  and  logical  methods  used  to 
assist  a  decision  maker  in  choosing  the  appropriate  decision  in  an  uncertain  world. 
Using  the  axioms  of  utility  theory,  mathematical  models  can  be  created  that  best 
describe  the  preferences  of  the  decision  maker  [18].  These  preferences  include  the 
decision  maker’s  trade-offs  between  different  value  measures  and  the  decision  maker’s 
probabilistic  preference  between  different  outcomes  [18].  Decision  analysis  can  apply 
to  both  single  attribute  situations  as  well  as  multiple  attribute  situations. 

Some  decision  analysis  practitioners  prefer  to  treat  multiple  attribute  applications 
in  a  way  that  examines  each  attribute  separately  in  terms  of  the  decision  maker’s 
preferences  for  that  attribute.  Marginal  utility  functions  are  developed  for  each  at¬ 
tribute;  these  separate  functions  are  then  combined  into  a  single  multiattribute  utility 
function  based  on  certain  assumptions  made  about  preferential  independence,  utility 
independence  and  additive  independence  amongst  the  attributes  [23]. 

Other  decision  analysis  practitioners  prefer  to  formulate  a  value  function  directly 
considering  the  deterministic  trade-offs  between  the  multiple  attributes.  A  utility 
function  is  then  assessed  for  this  value  measure  [1].  Matheson  and  Abbas  propose 
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the  idea  of  utility  transversality  in  a  utility  function  assigned  over  a  value  function 
[31].  This  concept  relates  the  risk  aversion  functions  of  the  individual  attributes  to 
the  value  trade-off  functions  between  these  attributes.  Abbas  also  relates  a  decision 
maker’s  risk  aversion  over  value  to  the  decision  maker’s  multiattribute  risk  aversion 
[2]- 

Often,  a  decision  maker  is  interested  in  adjusting  a  set  of  controllable  inputs  to 
optimize  a  set  of  outputs.  Response  Surface  Methodology  (RSM)  is  a  set  of  statistical 
and  mathematical  methods  used  to  develop  or  improve  processes  [32],  Used  widely  in 
industry,  most  RSM  applications  optimize  a  response  variable  which  is  a  function  of 
one  or  more  input  variables.  These  functions  can  be  known  exactly  such  as  through 
a  chemical  or  engineering  process.  Other  times,  the  underlying  function  is  not  known 
and  is  estimated  using  various  statistical  methods  [32] . 

In  many  cases,  more  than  one  response  variable  is  important  to  a  process.  One 
example  is  a  machining  process  with  machining  parameters  as  input  variables  and  two 
response  variables:  removal  rate  and  surface  roughness.  The  optimum  decision  might 
be  the  proper  combination  of  machining  parameters  that  simultaneously  maximizes 
removal  rate  and  minimizes  surface  roughness  [35]. 

Numerous  multiple  response  optimization  models  exist  in  current  RSM  literature 
[15,  9,  13,  24,  45].  Harrington  introduces  one  such  model  [15].  Harrington’s  desirabil¬ 
ity  function  transforms  each  response  to  an  individual  desirability  level  between  zero 
and  one.  The  optimum  strategy  is  then  the  one  which  produces  the  highest  geometric 
mean  of  the  individual  desirability  levels  [15]. 

Derringer  and  Suich  modified  the  form  of  the  individual  desirability  function  to 
be  more  flexible  [13].  They  change  the  individual  desirability  function  to  provide  the 
decision  maker  more  control  over  how  quickly  desirability  moves  from  zero  to  one  as 
the  response  moves  from  its  worst  value  to  its  target  value. 
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In  employing  the  desirability  function,  care  must  be  taken  by  the  analyst  to 
choose  parameters  consistent  with  the  decision  maker’s  own  preferences.  The  de¬ 
sirability  function  exhibits  implicit  and  explicit  assumptions  regarding  risk  attitude, 
value  trade-offs,  and  attribute  independence.  Potential  problems  may  result  from 
using  a  multiple  response  optimization  model  without  fully  understanding  these  as¬ 
sumptions.  Kros  and  Mastrangelo  introduce  the  idea  of  applying  utility  theory  to 
examine  the  assumptions  and  preferences  underlying  desirability  functions  [29].  They 
attempt  to  compare  and  contrast  assumptions  regarding  risk  preferences,  trade-offs, 
and  relationships  between  the  multiple  attributes  inherent  in  Derringer  and  Suich’s 
desirability  function  [29]. 

This  thesis  examines  the  desirability  function  from  a  decision  analysis  perspective. 
This  analysis  provides  knowledge  about  how  best  to  employ  the  desirability  function 
in  a  manner  consistent  with  the  decision  maker’s  value  trade-offs  and  risk  attitude. 

In  the  current  climate  of  budgetary  constraints  across  industrial  and  governmental 
organizations,  proper  analysis  of  a  decision  situation  is  vital  to  maximizing  the  limited 
resources  available.  When  confronted  with  a  situation  where  multiple  objectives  need 
to  be  considered,  the  analyst  needs  to  understand  the  assumptions  inherent  to  the 
model  chosen  so  that  it  is  consistent  with  the  decision  maker’s  preferences  and  the 
proper  solution  for  the  organization  can  be  found. 

The  rest  of  this  thesis  is  organized  as  follows.  Chapter  2  presents  a  review  of  rele¬ 
vant  literature.  Chapter  3  develops  the  methodology  used  to  examine  the  desirability 
function  and  the  methodology  used  to  find  a  robust  optimum  solution  from  a  decision 
analysis  perspective.  Chapter  4  presents  the  analysis  of  a  robust  optimization  solution 
in  a  wire  bonding  process  experiment  in  the  semiconductor  industry  using  various  as¬ 
sumptions  about  risk  attitude  and  value  trade-offs.  Chapter  5  presents  the  analysis  of 
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a  robust  optimization  solution  in  a  nation-building  (i.e. ,  SSTRO)  example.  Chapter 
6  offers  significant  findings,  recommendations,  and  suggestions  for  future  work. 
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II.  Review  of  Related  Literature 


2.1  Organization 

This  chapter  reviews  the  related  literature  applicable  to  the  thesis.  Section  2.2 
gives  an  overview  of  decision  analysis  including  risk  preferences  and  modeling  uncer¬ 
tainty  in  cases  of  single  attributes  and  multiple  attributes. 

Section  2.3  discusses  Response  Surface  Methodology  and  how  the  desirability  func¬ 
tion  is  used  to  find  an  optimum  setting  in  a  multiple  response  situation.  The  idea  of 
a  robust  optimum  point  in  a  noisy  environment  is  also  discussed. 

Section  2.4  discusses  stability  operations  and  their  place  in  the  United  States 
security  plan  are  then  discussed.  The  Air  Force  Research  Lab  models  such  operations 
with  their  National  Operational  Environment  Model. 

2.2  Decision  Analysis 

In  a  system  of  unknowns  such  as  a  business  venture  where  future  profits  and 
product  demand  growth  can  only  be  estimated,  decision  analysis  is  an  excellent  tool 
to  balance  the  factors,  both  certain  and  uncertain,  that  apply  to  a  given  decision 
situation.  Ronald  Howard  describes  decision  analysis  as  a  cycle  encompassing  deter¬ 
ministic,  probabilistic,  and  informational  phases  to  settle  on  a  logically  best  decision 
[18].  The  deterministic  phase  establishes  certain  relationships  between  the  variables 
within  the  problem.  The  probabilistic  phase  introduces  the  uncertainties  and  risk 
preference  of  the  decision  maker.  The  informational  phase  determines  the  value  of 
gathering  more  information.  If  more  information  is  to  be  gathered,  the  cycle  is  re¬ 
peated  until  no  new  information  is  deemed  necessary  by  the  decision  maker  [18]. 

The  uncertainty  involved  in  a  business  investment  venture  is  not  something  easily 
measured.  Howard  describes  two  types  of  probabilities:  objective  and  subjective.  An 
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objective  probability  is  one  that  is  measured  after  several  instances  of  the  uncertain 
event  occur.  For  example,  finding  after  1,000  coin  flips  that  the  coin  came  np  heads 
approximately  500  times  could  lead  one  to  objectively  assign  a  50%  probability  to  that 
coin  coming  np  heads  on  the  next  coin  flip.  Alternatively,  a  subjective  probability  is 
one  which  is  assigned  based  on  individual  knowledge  about  the  nature  of  the  uncertain 
event.  For  example,  assigning  a  50%  probability  to  a  coin  one  has  never  seen  flipped 
before  based  on  the  fact  the  coin  appears  to  be  ’fair’  is  subjective  [18].  In  most  cases, 
a  particular  business  venture  cannot  be  tried  several  times  to  see  how  likely  it  is  to 
be  successful.  Only  a  subjective  probability  can  be  assigned  to  its  success  based  on 
analytical  knowledge  of  the  nature  of  this  and  similar  ventures. 

2.2.1  Utility  Theory. 

Howard  describes  utility  theory  as  encompassing  five  axioms  concerning  the  idea 
of  lotteries  [18].  A  lottery  is  a  set  of  outcomes  in  which  exactly  one  occurs.  The 
first  axiom  requires  transitivity  in  preference.  If  three  alternatives,  A,  B,  and  C,  are 
available  and  if  an  individual  prefers  A  to  B  and  prefers  B  to  C,  this  individual  must 
prefer  A  to  C. 

The  second  axiom  calls  for  a  probabilistic  preference.  Consider  one  who  prefers  A 
to  B  to  C.  Then,  a  preference  probability,  p,  must  exist  where  this  person  is  indifferent 
to  accepting  B  for  certain  or  accepting  a  lottery  which  produces  A  with  probability 
p  or  produces  C  with  probability  (1  —  p).  When  a  particular  value  of  p  is  found,  B 
becomes  the  certain  equivalent  (CE)  of  the  lottery  between  A  and  C.  Figure  1  shows 
how  this  lottery  and  its  CE  are  usually  depicted  in  decision  analysis  literature. 

The  third  axiom  involves  substitution.  The  CE  of  a  lottery  can  be  exchanged  or 
substituted  for  the  lottery  itself  in  any  situation  without  any  changes  of  preference 
for  the  decision  maker. 
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A 
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Figure  1.  B  is  the  certain  equivalent  of  the  lottery  between  A  and  C 

The  fourth  axiom  states  the  acceptance  of  probability  as  the  means  to  describe 
uncertainty  and  also  expresses  indifference  in  the  ordering  of  lotteries  in  the  decision 
at  hand.  It  states  that  multiple  levels  of  lotteries  may  be  replaced  by  a  single  lottery 
with  the  prizes  and  associated  probabilities  calculated  by  the  laws  of  probability.  In 
essence,  this  axiom  states  that  the  lottery  itself  holds  no  intrinsic  value  to  the  decision 
maker. 

The  fifth  axiom  states  if  a  decision  maker  is  faced  with  two  lotteries,  each  with 
outcomes  of  A  or  B,  and  the  decision  maker  prefers  A  to  B,  the  decision  maker  must 
prefer  the  lottery  that  yields  A  with  the  higher  probability. 

2.2.2  Risk  Preference. 

Any  practical  decision  opportunity  which  involves  uncertainty  must  take  into  ac¬ 
count  the  decision  maker’s  preference  towards  risk.  Howard  points  out  that  few  people 
would  be  willing  to  give  up  next  year’s  salary  for  a  50%  chance  of  doubling  it  versus 
receiving  no  salary  for  the  year  although  this  represents  a  fair  proposition  [18].  A 
person  indifferent  to  the  preceding  deal  is  considered  to  be  risk  neutral.  Howard  sug¬ 
gests  that  most  people  and  organizations  are  risk-averse.  A  risk-averse  individual’s 
CE  for  a  given  lottery  is  less  than  the  expected  value  of  the  outcome  of  that  lottery 
[18]- 

8 


2.2.3  Utility  Function. 


One  way  to  calculate  a  decision  maker’s  risk  preference  is  to  find  the  preference 
probabilities  for  all  feasible  alternatives  within  the  current  decision  space.  Depending 
on  the  situation  and  the  number  of  alternatives,  this  can  become  quite  cumbersome. 
The  utility  function  encodes  the  decision  maker’s  risk  preference.  This  function  as¬ 
sociates  an  outcome  from  a  lottery  or  other  uncertain  situation  with  a  utility  value. 
Moreover,  the  decision  maker’s  utility  of  a  lottery  is  the  expected  value  of  the  u-values 
of  the  lottery’s  outcomes.  From  the  example  in  Figure  1,  the  decision  maker’s  utility 
for  the  lottery  would  be  U\  =  p*U  (A)  +  (l—p)*U  ( C ).  If  the  decision  maker  prefers 
one  lottery  to  another,  the  preferred  lottery’s  utility  will  be  higher  than  the  other. 

Howard  also  points  out  that  although  these  u-values  can  model  the  decision 
maker’s  preferences  among  various  alternatives  or  lotteries,  the  actual  magnitude 
of  a  utility  means  nothing  on  its  own  [18].  Comparing  utilities  cannot  be  used  to 
show  how  strongly  one  alternative  is  preferred  to  another.  It  can  only  be  used  for 
ranking  purposes  [18]. 

Arrow  and  Pratt  [6,  37]  introduce  the  risk  aversion  function, 


7  (y)  = 


u"{y) 

u'(y)  ' 


They  show  the  negative  ratio  of  a  utility  function’s  second  derivative  over  its  first 
derivative  correctly  measures  the  local  risk  aversion  at  any  given  point  along  a  utility 


function.  This  measure  assumes  that  the  utility  function  in  question  is  monotonically 
increasing  and  is  twice  differentiable.  If  7 (y)  =  0,  the  utility  function  is  describing 
risk  neutral  behavior.  If  7 (y)  >  0,  it  is  describing  risk  averse  behavior  and  if  7 (y)  <  0, 
it  is  describing  risk  seeking  behavior  [37]. 
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2.2.4  Modeling  Uncertainty. 


Decision  analysts  employ  several  methods  to  model  uncertainty.  Two  common 
methods  are  using  a  discretized  approximation  of  the  continuous  distribution  or  us¬ 
ing  a  Monte  Carlo  simulation  over  the  distribution  [11].  A  discretized  approximation 
simplifies  the  calculations  involved  with  using  a  continuous  function.  When  the  con¬ 
tinuous  probability  function  is  unknown,  it  can  be  estimated  by  using  known  data  to 
estimate  the  parameters  of  a  Beta  distribution. 

The  Beta  distribution  is  a  flexible  continuous  probability  distribution  given  over 
a  set  bounded  range  [3].  Equation  2.2  shows  the  density  function. 


Beta{a,  (3,  a,  b,  x )  = 


( x  —  a)a  1(6  —  x)P  1 
f^(x  —  a)a~l(b  —  x)P~1dx 


(2.2) 


Where  a  and  b  are  the  lower  and  upper  bounds  of  the  domain,  and  a  and  /3  are 
the  two  parameters  of  the  Beta  distribution  [3]. 

Abbas  et  al.  [3]  describe  assessing  the  data  into  fractiles  and  then  using  Matlab’s 
fminsearch  function  to  estimate  the  two  parameters  (a,  /3)  by  minimizing  the  squared 
errors  from  the  value,  x,  from  the  inverse  of  its  related  probability  p.  Equation  2.3 
shows  the  relevant  expression, 


minV(Xi-W)2,  (2.3) 

where  Xt  =  Betalnverse(pi,  a,  f3,  a,  b),  Xt  is  the  midpoint  of  the  i-th  fractile,  n  is  the 
number  of  fractiles  used,  and  a,  (3  >  0.  The  parameters  a  and  b  are  chosen  by  the 
user  and  pt  is  the  probability  associated  with  the  i-th  fractile  [3] . 

Monte  Carlo  simulation  is  a  method  to  model  the  uncertainty  in  a  given  system. 
Assume  an  uncertainty  can  be  modeled  by  a  known  distribution;  a  computer  then 
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produces  a  large  number  of  random  numbers  from  that  distribution.  The  utilities  of 
the  associated  outcomes  are  then  used  to  find  the  expected  utility  of  the  lottery  [11], 
Often  multiple,  correlated  random  variables  must  be  modeled  in  the  decision 
model.  Sklar  [44]  states  given  a  joint  cumulative  distribution  function  (CDF) 
F(x i, . . .  ,xn)  with  marginal  CDFs  F\(xi), . . . ,  Fn(xn),  the  joint  CDF  can  be  writ¬ 
ten  as  a  function  of  its  marginals, 

F(x i, . . . ,  xn)  =  C[Fi(xi), . . . ,  Fn{xn)\.  (2.4) 

The  function,  C,  is  called  a  copula.  If  each  marginal  CDF,  Fil  is  continuous,  then  C 
is  a  unique  function  for  the  given  joint  CDF  [44], 

Clemen  and  Reilly  [10]  describe  how  to  create  the  joint  multivariate  random  sam¬ 
ple  for  a  Monte  Carlo  simulation  using  the  multivariate  normal  copula:  1)  generate 
a  vector  of  random  numbers  (yi, ... ,  yn)  from  a  multivariate  normal  random  number 
generator  using  the  desired  correlation  matrix,  R.  2)  the  standard  normal  distribu¬ 
tion  function,  $(yj)  is  calculated  for  each  of  the  n  variables,  3)  the  inverse  marginal 
distribution  functions  for  each  variable  is  then  used  to  calculate  the  vector  of  required 
variates,  (F1”1[$(|/i)], . . . ,  F'“1[<F(|/n)])  for  the  Monte  Carlo  simulation. 

2.2.5  Multiattribute  Utility  Theory. 

Two  common  methods  for  formulating  multiattribute  utility  functions  include 
creating  a  multiattribute  value  function  using  deterministic  trade-offs  between  the 
attributes  and  then  assessing  a  single  attribute  utility  function  over  that  value  or 
by  assessing  conditional  utility  functions  over  each  of  the  individual  attributes  and 
then  combining  these  into  a  single  multiattribute  utility  function  by  using  various 
independence  assumptions  [1]. 
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Some  of  the  independence  assumptions  made  while  forming  a  multiattribute  utility 
function  include  preferential  independence,  utility  independence,  and  additive  inde¬ 
pendence.  To  describe  these  independence  assumptions,  consider  a  multiattribute 
decision  situation  with  up  to  three  attributes,  yi,  y2,  y 3  where  each  attribute  falls 
within  the  range,  yi  G  [y -J ,  y* ] ,  i  =  1,2,3.  Assume  the  attributes  can  be  ordered  such 
that  (2/° ,  2/3)  is  !he  least  preferred  prospect  and  (y]]\y2,y3)  is  the  most  preferred 

prospect. 

The  attributes,  yi  and  y2,  are  said  to  be  preferentially  independent  of  y3  if  a 
given  deterministic  prospect  {yfK  y2\  y3)  is  preferred  to  or  indifferent  to  (y^,  y£\  y3) 
regardless  of  the  value  of  y3  [22],  Equation  2.5  displays  this  concept. 

(y[2\y(22\y3)  h  {y{i\y2\y: s),Vy3  (2.5) 

In  a  two  attribute  decision  space  with  attributes,  y,  and  y2,  y\  is  said  to  be  utility 
independent  of  y2  if  the  conditional  preferences  of  y\  given  a  certain  value  of  y2  do  not 
depend  on  the  value  of  y2  [22],  In  other  words,  the  conditional  utility  function  over  y\ 
given  a  fixed  value  of  y2  is  a  positive  linear  transformation  of  the  conditional  utility 
function  over  y\  given  y2  is  hxed  at  any  other  value  [22],  If  y\  is  utility  independent 
of  y2,  then  the  two-attribute  utility  function  must  be  of  the  form  in  Equation  2.6, 

u(yi,y2)  =  g(y2)  +  h(y2)u(y1,  y21}),  (2.6) 

where  y(y2)  and  h(y2)  are  both  functions  that  depend  on  y2  only  and  w(yi,  y2 )  is  the 
conditional  utility  function  over  y\  given  y2  =  [23]. 

Two  attributes,  y\  and  y2,  are  said  to  be  additive  independent  if  the  comparison 
between  any  two  prospects  depends  only  on  the  marginal  preference  structure  of  the 
two  attributes.  Consider  two  levels  of  each  attribute,  y[x\  y[2\  y^\  y^\  If  y\  and  y2 
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are  additive  independent,  then  the  lottery  with  equal  chances  of  the  two  prospects, 
(Ui^i  U^)  an(l  {lh  \  V-P)  is  equivalent  to  the  lottery  with  the  two  prospects,  (y^\  y^) 
and  (y[2),  y^),  with  equal  chances  [23]. 

If  two  attributes  are  additive  independent,  the  two- attribute  utility  function  is 
additive  and  can  be  written  as  in  Equation  2.7. 

«(yi,y2)  =  kyiUyi(yi)  +  kmuv  2(2/2)  (2-7) 

The  utility  function  is  normalized  such  that  u^y^y®)  =  0  and  ^(y^y^)  =  1.  The 
marginal  utility  function,  uyi(yi),  is  normalized  such  that  uyi(y^)  =  0  and  uVi(y*)  =  1 
(?'  =  1,2).  The  constants,  kyi  and  km  are  calculated  such  that  kyi  =  uiy^y®)  and 

K2  =u(Vi,y 2)  [23]. 

Richard  [39]  dehnes  multivariate  risk  aversion  as  the  condition  in  which  the  de¬ 
cision  maker  prefers  a  lottery  with  an  even  chance  of  the  prospects,  (y^\y^)  or 
(yf^y^),  where  y\ 1 <  yf  \  i  =  1,2  to  a  lottery  with  even  chances  for  (yf^y^)  or 
(y)  ;).  Indifference  between  these  two  lotteries  is  referred  to  as  multivariate  risk 

neutrality.  A  preference  of  the  second  lottery  to  the  first  is  multivariate  risk  seeking 
behavior.  Richard  shows  that  the  sign  of  the  mixed  partial  derivative  of  the  utility 
function  indicates  the  multivariate  risk  preference  expressed  by  the  utility  function 
[39].  Table  1  gives  the  results. 

Table  1.  Sign  of  utility  function’s  mixed  partial  derivative  compared  to  multivariate 
risk  preference 

Sign  of  Multivariate 

o^-U(yi,y2)  Risk  Preference 

averse 

0  neutral 

+  seeking 
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2.2.6  Utility  Transversality. 


Matheson  and  Abbas  introduce  the  concept  of  utility  transversality  in  the  multi¬ 
attribute  case  where  a  utility  function  is  assigned  over  a  multiattribute  value  function 
[31].  Equation  2.8  shows  the  general  formulation  of  such  a  utility  function, 


U{yi,...,yn)  =  Uv{V{yu...,yn)),  (2.8) 

where  yi,...,yn  are  the  attributes  under  consideration,  V(yi, . . . ,  yn)  is  the  deter¬ 
ministic  value  function  over  these  attributes,  and  Uy(V)  is  the  utility  function  with 
respect  to  value  [31]. 

Assuming  that  the  utility  and  value  functions  are  both  monotonically  increasing 
and  twice  differentiable,  the  risk  aversion  function  with  respect  to  a  single  attribute, 
y%,i  —  n,  is  given  in  Equation  2.9, 


1U  = 
'  Vi 


K 

UL 


(2.9) 


where  U"  =  d2U (yu  . . . ,  yn)/dy2  and  U'y.  =  dU {yu  . . . , yn)/dyi  [31]. 

Considering  the  two  dimensional  case  where  U(yi,y2)  =  Uy(V(yi,  y2))  and  fol¬ 
lowing  the  chain  rule  of  differentiation,  Matheson  and  Abbas  [31]  show  that  the  risk 
aversion  with  respect  to  a  single  variable  is 


7 


u 

yi 


(2.10) 
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The  risk  aversion  with  respect  to  value  is  7 y.  The  partial  derivative  of  value  with 
respect  to  y\  is  IT  =  dV(yi,y2) /dyi.  Matheson  and  Abbas  define  7^  as  the  value 
function’s  contribution  to  the  risk  aversion  with  respect  to  y\  as 


V” 

v  =  _v_yi 
iy  1  Y' 

yi 


(2. IT) 


where  V”x  =  d2V (yi,  y2) / dyf  [31].  This  is  analogous  to  the  definition  of  risk  aversion 
shown  in  Equation  2.1. 

The  risk  aversion  with  respect  to  a  single  attribute  is  defined  completely  by  the 
risk  aversion  with  respect  to  value  and  the  form  of  the  deterministic  value  function 
itself.  Since  the  value  function  is  deterministic  and  can  be  assessed  by  specifying  the 
deterministic  tradeoffs  between  attributes,  then  assessing  the  utility  function  over 
value  determines  the  risk  aversion  function  for  all  attributes  [31]. 

Matheson  and  Abbas  define  the  utility  transversality  relation  as  the  relationship 
between  the  risk  aversion  functions  of  the  various  attributes  and  is  shown  in  Equation 


2.12  [31]. 


=  +  (2-12) 

The  value,  t,  is  the  deterministic  tradeoff  function  between  the  attributes  y\  and  y2 
along  an  isopreference  contour,  calculated  as  in  Equation  2.13  [31]. 


t(yi,V2) 


V' 
y  1 

V' 

2/2 


dy  2 

dyi 


|  isopreference 


contour 


2.2.7  Attribute  Dominance. 


(2.13) 


Abbas  and  Howard  introduced  the  concept  of  attribute  dominant  utility  functions 
[4],  This  special  class  of  multiattribute  utility  functions  satisfies  four  conditions. 
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Consider  a  two-attribute  decision  situation  where  a  given  prospect  can  be  written 
as  (2/1 , 2/2)  and  the  two  attributes,  y\  and  y2,  fall  within  the  respective  ranges,  y\  G 
[2/1 , 2/i]  and  V2  G  [2/2  5  2/2 ]  •  Assume  the  attributes  can  be  ordered  such  that  (2/1,2/°)  is 
the  least  preferred  prospect  and  (2/*,  2/2)  is  ^ie  m°st  preferred.  Assume  the  attributes 
are  mutually  preferentially  independent  and  the  prospects  are  arranged  such  that  for 
any  y^\  y^\  y^\  y£\  the  following  conditions  hold  [4], 


y?)  >  y[1]  =>  (2/i2),  2/2)  h  (2/i1},  2/2)V2/2  e  [2/2, 2/2]  and 

y(2]  >  y{2]  =>  (2/1, 2/22))  h  (2/1,  2/21})V2/i  e  [y^yl]  (2.14) 

Assume  a  multiattribute  utility  function  Uyiy2(y\,  y2)  exists  with  range  as  given 
in  Equation  2.15  [4], 

o  <  uyiy2(y1,y2)  <  l,Vj/i  G  [yi,y*],y2  e  [y02,y2}  (2.15) 

Based  on  this  formulation,  the  utility  of  the  prospects,  (y®,  y%)  and  (y{,y2),  are  as 
shown  in  Equation  2.16  [4]. 


umK(y1,v°)  =  o.  unvM,yl)  =  i  (2.16) 

With  the  preceding  assumptions,  an  attribute  dominance  utility  function  is  one 
where  a  prospect,  (yi,  y2),  is  a  least  preferred  prospect  if  at  least  one  of  its  attributes, 
yi  or  y2,  is  at  its  minimum  value.  Equation  2.17  describes  this  condition  [4]. 
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^S/l?/2  (2/1 5  2/2  )  EWyi,Jfa)  Uy1y2  (j)li  1/2) 

Vj/i  e  [j/?,j/J],y2  e  [2/2, 2/2] 


(2.17) 


2.3  Response  Surface  Methodology 

Response  Surface  Methodology  (RSM)  is  a  set  of  mathematical  and  statistical 
tools  used  to  optimize  systems  [33].  RSM  provides  techniques  to  estimate  a  model 
of  the  system  of  interest  by  running  an  experiment  varying  the  input  variables  of 
interest  and  measuring  the  response  of  interest  [32],  A  function  is  estimated  from  the 
results  of  such  an  experiment  as  shown  in  Equation  2.18 

y  =  /3x  +  e  (2-18) 

where  y  is  the  response  in  question,  x  is  the  vector  of  input  variables,  (3  is  a  vector 
of  coefficients  estimated  from  the  experiment  and  e  is  a  random  variable  representing 
the  random  noise  within  the  system  [32], 

Equation  2.18  can  be  extended  to  multiple  responses  using  a  vector  of  responses, 
y,  a  matrix  of  coefficients,  B,  and  a  vector  of  random  noise  variables,  e.  Equation 
2.19  shows  such  an  equation. 


y  =  Bx  +  e  (2.19) 

Classic  RSM  and  regression  techniques  are  built  around  systems  occurring  in 
nature  or  in  man-made  environments.  Special  considerations  must  be  given  when  an¬ 
alyzing  computer  simulation  models.  Common  random  numbers  are  commonly  used 
when  comparing  different  simulation  scenarios  since  they  can  sharpen  the  compari- 
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son  between  scenarios.  When  attempting  to  estimate  the  simulation  model  with  a 
regression  polynomial,  non-overlapping  pseudo-random  number  (PRN)  streams  must 
be  used  at  the  different  design  points  so  the  outputs  are  independently  identically 
distributed  as  RSM  methods  assume  [27]. 

One  other  difference  in  a  computer  simulation  experiment  is  that  it  can  be  ac¬ 
complished  sequentially  without  needing  to  consider  blocking  the  the  analysis.  In  a 
classic  experiment,  the  experimenter  would  generally  randomize  the  order  of  the  de¬ 
sign  points  to  remove  any  ordering  bias.  If  an  augmented  design  were  to  be  added  to 
this  experiment,  the  experimenter  would  have  to  analyze  the  data  with  that  sequence 
in  mind  since  the  augmented  design  points  were  not  randomized  with  the  original 
set.  A  computer  simulation  experiment  can  be  sequenced  similarly  without  any  ad¬ 
ditional  concern  to  the  change  in  analysis  due  to  the  augmented  design.  The  order 
of  the  experiment  does  not  affect  the  computer  output  if  a  non-overlapping  random 
stream  of  PRNs  is  used  to  initiate  any  design  points  [27]. 

2.3.1  Desirability  Function. 

Harrington  introduces  the  desirability  function  in  order  to  find  an  optimal  solu¬ 
tion  when  optimizing  multiple,  often  competing,  objectives  simultaneously  [15].  The 
desirability  function  scales  each  objective  measure,  to  a  scale  between  0  and  1, 
denoted  by  dt .  The  overall  desirability  of  a  given  solution  is  the  geometric  mean  of 
the  individual  desirabilities  as  shown  in  Equation  2.20  [15].  The  optimal  solution  is 
the  vector  of  yds  that  produces  the  largest  overall  desirability,  D. 

D  =  (<M2  •••<**)"  (2.20) 
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Derringer  and  Suich  modify  the  function  for  the  individual  objective  desirabilities 
so  that  they  are  more  flexible  [13].  Equation  2.21  shows  the  function  for  an  individual 
objective’s  desirability  where  a  maximum  value  of  the  response,  is  desired. 


di 


0, 


Vi  Li 

Ti-Li 


Vi  —  Lj 

i  Li  <  y%  <  Ti 


1,  Vi  >  Ti 


(2.21) 


The  parameters,  Li:  T),  and  rt ,  are  provided  by  the  decision  maker.  The  param¬ 
eter  L,  indicates  the  minimum  acceptable  value  of  yi.  Any  value  of  yi  below  Li  is 
unacceptable.  This  qualitative  characteristic  is  expressed  in  the  desirability  function 
by  forcing  di  —  0  which  then  forces  the  overall  desirability  to  D  =  0  as  well.  The 
parameter  T%  is  an  optimal  value  of  yi  or  the  point  at  which  any  more  yi  would  give 
no  additional  value.  The  parameter  r,  controls  how  quickly  the  individual  objective 
desirability  increases  from  0  to  1  as  yi  increases  from  L*  to  Ti.  Figure  2  depicts  how 
different  values  of  rt  affect  the  desirability  function. 


Figure  2.  Maximizing  desirability  functions,  di ,  for  various  levels  of  r,  [13] 
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Equation  2.22  shows  the  function  for  an  individual  objective’s  desirability  where 


a  minimum  value  of  the  response,  y *,  is  desired. 


<u=  < 


Vi-Uj 

Tt-Ui 


0, 


Vi  <  Ti 

Ti  <  Vi  <  Ui 
Vi  >  Ui 


(2.22) 


The  parameters,  Ui,  Tt ,  and  rt,  are  similarly  provided  by  the  decision  maker.  The 
parameter  Ux  is  the  maximum  value  of  yt.  Any  value  above  this  is  unacceptable  to 
the  decision  maker  and  will  drive  the  overall  desirability  to  D  —  0.  The  parameter 
Ti  is  similar  to  its  use  in  the  maximizing  function.  The  parameter  T)  is  the  optimal 
value  of  yt  or  the  point  at  which  any  level  of  yt  below  this  would  warrant  little  extra 
value  [13]. 

At  times,  a  specific  target  value  is  desired.  In  this  case,  a  two-sided  transformation 
is  applicable  as  shown  in  Equation  2.23. 


0,  yt  <  Li 


dt={ 


Vi  Li 
Ti~Li 

Ui  Uj 
Ti-Ui 


Li  <  yi  <  Ti 


Ti  <  yi  <  Ui 


(2.23) 


0,  yt  <  Ui 


The  parameters,  L,  and  Sj,  are  synonymous  to  L,  and  rt  in  equation  2.21.  The 
parameters,  Ut  and  ti,  are  synonymous  to  Ut  and  rt  in  equation  2.22.  The  parameter 
Ti  is  the  optimal  value  of  the  objective,  yi.  When  yi  =  T%,  the  individual  desiribility, 
di  =  1  [13].  Figure  3  depicts  two-sided  transformations  for  various  levels  of  .s*  and  ti. 
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L 


T 


U 


Figure  3.  Two-sided  desirability  functions,  d*,  for  various  levels  of  Si  and  ti  [13] 
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2.3.2  Robust  Optimization. 


The  optimal  solution  found  in  a  multiple  response  situation  using  the  desirabil¬ 
ity  function  does  not  take  uncertainty  into  consideration  and  does  not  consider  the 
variability  of  the  responses  [36]. 

Figure  4  illustrates  the  concept  of  a  robust  solution.  The  x-axis  shows  an  input 
variable  affecting  a  maximum  response  along  the  y-axis.  At  x  —  X2,  point  B  is  clearly 
the  global  maximum  within  this  range.  However,  as  x  is  allowed  to  vary  from  x2  by 
r,  the  response  drops  A#,  from  y 2  to  y'2.  Conversely,  if  the  local  optimum  point  A  is 
considered  and  x  is  allowed  to  vary  from  x\  by  r,  the  response  drops  A^,  from  iji  to 
y\ .  In  this  situation,  where  A^  <  A#,  point  A  can  be  considered  more  robust  than 
point  B  [16]. 


Figure  4.  Illustration  of  Robust  Solution  [16] 

From  a  decision  analysis  perspective,  the  preference  between  point  A  and  point 
B  is  dependent  upon  the  decision  maker’s  risk  preference.  A  sufficiently  risk  averse 
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decision  maker  would  prefer  point  A.  A  decision  maker  tending  toward  risk  neutral 
or  risk  seeking  behavior  would  prefer  point  B  instead. 

Taguchi  introduces  robust  parameter  design  [46,  47]  for  situations  where  uncon¬ 
trollable  or  nuisance  factors  exist  and  a  solution  that  is  insensitive  to  the  variability 
of  these  nuisance  factors  is  desired. 

Vining  and  Myers  present  a  constrained  optimization  method  using  predictive 
regression  models  for  both  the  process  mean  and  variance  [51].  The  robust  solution  is 
found  by  either  constraining  the  mean  while  minimizing  the  variance  or  constraining 
the  variance  while  optimizing  the  mean  [51]. 

While  these  two  methods  focus  on  a  single  response  of  interest,  Peterson,  Miro- 
Quesado  and  del  Castillo  present  a  robust  approach  to  a  multiple  response  problem 
using  Monte  Carlo  simulation  [36].  After  calculating  the  response  equations  to  the 
seemingly  unrelated  regression  (SUR)  model,  they  calculate  the  probability  a  given 
set  of  inputs  would  give  an  acceptable  set  of  output  values  using  results  from  a  Monte 
Carlo  simulation. 

2.4  Stability,  Security,  Transition,  and  Reconstruction  Operations 

The  National  Security  Strategy  (NSS)  [34]  outlines  the  overall  security  strategy  for 
the  nation.  President  Obama  states  the  greatest  threat  to  the  American  people  is  the 
possibility  of  violent  extremists  obtaining  weapons  of  mass  destruction.  Engagement 
with  other  countries,  especially  emerging  nations,  is  key  to  the  NSS  [34],  President 
Obama  notes  the  need  for  an  interagency  approach  to  this  end.  Assisting  develop¬ 
ing  countries  join  the  world  economy,  manage  their  security  needs,  and  assist  their 
governments  to  lead  their  nations  with  an  eye  to  human  rights  is  imperative.  The 
military’s  role  includes  excelling  at  counterterrorism  and  stability  operations  while 
maintaining  readiness  to  address  the  full  range  of  military  operations  [34], 
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Stability  operations  are  essential  to  keeping  a  safe  and  secure  environment  for 
U.S.  interests  abroad.  Department  of  Defense  Instruction  (DoDI)  3000.05  [48]  states 
stability  operations  are  a  core  U.S.  military  mission  and  should  be  conducted  across 
the  range  of  military  operations.  Further,  U.S.  military  operations  must  strike  “ap¬ 
propriate  balance  between  offensive  and  defensive  operations  and  stability  operations 
in  all  phases”  [49]. 

Military  stability  operations  are  planned  based  on  effects  based  operations  (EBO) 
as  outlined  by  the  United  States  Joint  Forces  Command  [50].  A  nation  or  region  in 
question  is  characterized  as  a  system  of  systems  in  terms  of  its  political,  military, 
economic,  social,  infrastructure,  and  information  (PMESII)  systems.  The  effects  of 
diplomatic,  informational,  military  and  economic  (DIME)  instruments  of  national 
policy  is  then  modeled  to  estimate  the  PMESII  effects  within  the  nation  or  region  in 
question  [50]. 

The  current  NSS  is  interested  in  developing  diplomatic  relations,  security  mea¬ 
sures,  and  economic  markets  in  African  nations  [34],  The  U.S.  is  particularly  con¬ 
cerned  about  assisting  the  Democratic  Republic  of  Congo  (DRC)  in  its  efforts  to  create 
a  secure,  stable  government.  Concern  still  exists  about  human  rights  violations  in  the 
DRC  after  its  recent  presidential  election  [12]. 

2.5  National  Operational  Environment  Model 

The  National  Operational  Environment  Model  (NOEM)  is  a  simulation  model 
representing  a  nation-state  or  region  used  to  test  a  variety  of  courses  of  actions  [42], 
NOEM  subscribes  to  the  DIME  and  PMESII  paradigm  while  modeling  nation  security 
and  stability  [43].  The  main  contributions  to  NOEM  come  from  Richardson  [40], 
Robbins  [41],  and  Fensterer  [14].  Richardson  [40]  applies  system  dynamics  modeling 
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techniques  to  a  Stability  and  Reconstruction  effort  at  a  national  level.  This  allows  a 
statistical  analysis  of  various  high-level  policy  choices. 

Robbins  [41]  improves  upon  this  idea  by  developing  a  more  detailed  model,  the  Sta¬ 
bilization  and  Reconstruction  Operations  Model  (SROM),  which  allows  for  more  com¬ 
prehensive  analysis  of  the  nation-building  efforts.  SROM  models  DIME  and  PMESII 
interactions  at  the  sub-national,  regional  level  which  allows  testing  of  a  wide  variety 
of  policy  options  on  a  national  or  regional  basis. 

Fensterer  [14]  uses  a  Value  Focused  Thinking  strategy  to  capture  the  important 
values  of  nation  state  stability.  Using  Department  of  Defense  (DoD)  guidance,  Fen¬ 
sterer  suggests  five  fundamental  values  of  stability:  Economy,  Governance,  Rule  of 
Law,  Security,  and  Social  Well-Being. 

Based  on  this  research,  SROM  was  re-engineered  at  the  Air  Force  Research  Labo¬ 
ratory  to  become  NOEM  [42],  NOEM  contains  several  modules  to  model  the  complex 
and  critical  DIME  and  PMESII  interactions  within  a  region.  Although  most  modules 
contain  deterministic  models,  an  Agent-Based  model  represents  the  population  and 
provides  a  stochastic  element  to  the  overall  model  [42], 
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III.  Methodology 


3.1  Research  Methodology 

This  chapter  develops  the  methodology  used  to  examine  the  desirability  function 
and  the  methodology  used  to  find  a  robust  optimum  solution  from  a  decision  anal¬ 
ysis  perspective.  Section  3.2  an  analysis  of  the  desirability  function  from  a  decision 
analysis  perspective.  Section  3.2.1  analyzes  the  desirability  function  as  a  value  func¬ 
tion.  Sections  3.2.2  and  3.2.3  analyze  a  utility  function  assessed  over  the  desirability 
function.  Section  3.3  develops  the  methodology  taken  to  find  a  robust  decision  point 
within  a  multiple  response  system. 

3.2  Desirability  Function  Analysis 

This  section  presents  an  analysis  of  Derringer  and  Suich’s  desirability  function 
[13]  from  a  decision  analysis  perspective.  Two  different  cases  are  examined.  In  the 
first  case,  assume  the  desirability  function  is  a  value  function.  In  the  second  case, 
assume  the  desirability  function  is  a  utility  function.  The  analysis  focuses  on  the 
two-dimensional  case  with  responses,  y\  and  y2.  The  analysis  is  generalized  to  an 
n-dimensional  case,  which  is  presented  in  Appendix  A. 

The  desirability  function  for  the  2-dimensional  case  is 

D  =  (dic^)5.  (3.1) 

Consider  a  system  with  n  inputs,  (xi,. . .  ,xn).  Each  input,  ay  falls  within  the  ex¬ 
perimental  region  defined  by  ay  G  [a;°,a;*],  i  =  1 , ,n.  These  inputs  cause  changes 
in  two  responses,  ij\  and  y2  which  each  fall  within  the  range,  y*  G  [yf,  y*],  i  —  1,2. 
This  analysis  focuses  on  the  case  where  a  maximum  response  is  desired  as  displayed 
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in  Equation  2.21  with  no  loss  of  generality  since  the  other  forms  of  the  desirability 
function  can  be  transformed  into  this  case.  Only  the  non-trivial  piece  of  Equation 
2.21  where  L*  <  y*  <  T),  i  =  1,  2  is  considered.  Equation  3.2  displays  this  non-trivial 
piece. 


Hi  Li 
Ti  —  L{ 


Li  <  y%  <  Ti 


Where  [Lt,  T)]  C  [y-J,  y*],  i  =  1,2.  This  forces  0  <  D  <  1. 


(3.2) 


3.2.1  Desirability  function  as  value  function. 

Consider  the  case  in  which  the  desirability  function  is  used  as  a  value  function.  The 
desirability  function  can  be  used  as  a  value  function  if  the  decision  maker  assumes  the 
expected  responses  from  the  regression  equations  are  deterministic.  The  deterministic 
trade-off  between  the  two  attributes  can  then  be  examined  using  the  tradeoff  function 
as  defined  in  Chapter  II. 


D' 

t(vi,V2)=  V1 


D 


V2 


dy_t 

dx 


|  isopreference  contour 


(3,3) 


The  deterministic  tradeoffs  between  the  two  attributes  can  then  be  stated. 


T  i 

tivum)  =  — 
r2 


y2  —  L'2 


(3.4) 


,2/i  -  L  i 

Equation  3.4  indicates  how  many  units  of  y2  the  decision  maker  is  willing  to  give 
up  in  order  to  increase  ij\  by  one  unit,  at  the  point  (yi,y2).  In  this  case,  the  tradeoff 
is  a  ratio  of  the  two  exponents  rq  and  rq  multiplied  by  the  ratio  of  the  differences 
between  the  two  current  values  of  y%  and  their  respective  minimum  acceptable  values, 
Li.  Table  2  shows  how  the  tradeoff  changes  as  each  response  or  parameter  is  increased 
while  all  others  are  held  constant.  Interestingly,  the  two  target  values,  7)  and  T2, 
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cancel  out  of  the  function  when  the  ratio  of  the  two  partial  derivatives  of  the  overall 
desirability  function  is  calculated. 


Table  2.  Tradeoff  varies  with  respect  to  different  variables  increasing 


Variable  increasing  Effect  on 
(all  else  constant)  t(y1,y2 ) 

yi  decreases 

y2  increases 

7T  increases 

r2  decreases 

L\  increases 

L-2  decreases 


The  desirability  function  tends  to  move  the  responses  away  from  their  minimum 
values.  When  a  response  is  pushed  near  its  minimum  value,  its  individual  desirability, 
di,  tends  to  zero. 

For  a  constant  value  of  y2,  t(yi,y2)  decreases  as  y\  increases.  As  ij\  increases  from 
its  minimum,  d\  increases  from  zero  which  increases  the  overall  desirability.  The 
further  y\  is  from  its  minimum,  the  less  y2  a  decision  maker  is  willing  to  give  up  for 
further  increases  in  y\ . 

For  a  constant  value  of  y i,  t(yi,y2)  increases  as  y2  increases.  The  same  argument 
applies  as  above.  The  further  y2  is  from  its  minimum,  the  more  y2  a  decision  maker 
is  willing  to  give  up  for  further  increases  in  y\ . 

Consider  the  following  example  where  y2  is  10  units  above  its  minimum  value,  ij\ 
is  two  units  over  its  minimum,  and  ry  —  r2  =  1.  The  decision  maker  is  willing  to 
reduce  y2  by  five  units  to  increase  y\  by  one  unit.  See  Equation  3.5. 


t(L\  +  2,  L-2  +  10) 


1  no' 

1  ~2~ 


5 


(3.5) 


Now  let  y2  be  one  unit  above  its  minimum  value  not  10  units,  as  above  and  keep 
all  other  values  the  same.  The  decision  maker  is  only  willing  to  reduce  y2  by  0.5  units 


to  increase  y\  by  one  unit.  Equation  3.6  depicts  this  result.  After  y2  is  reduces  from 
10  to  1,  the  decision  maker  does  not  have  as  much  excess  y2  to  pay  in  order  to  increase 
y\ .  The  decreased  level  of  y2  results  in  a  lower  tradeoff  between  the  two  attributes. 


t(L i  +  2,  L2  +  1) 


1  f1 
1  2 


0.5 


(3.6) 


Increasing  rq  increases  the  trade-off.  As  rq  increases,  d\  decreases  for  a  given  value 
of  |/i .  A  higher  rq  implies  the  higher  values  of  y±  are  worth  a  premium;  the  decision 
maker  is  willing  to  decrease  more  of  y2  in  order  to  increase  y\ .  A  smaller  ri  allows  a 
much  lower  y\  to  achieve  higher  desirability. 

Increasing  r2  decreases  the  tradeoff.  As  r2  increases,  d2  decreases  for  a  given  value 
of  y2.  A  higher  r2  implies  the  higher  values  of  y2  are  worth  a  premium;  the  decision 
maker  is  willing  to  decrease  y2  less  in  order  to  increase  y\ . 

Increasing  Li  increases  the  tradeoff.  The  minimum  level  of  yi  is  being  increased, 
forcing  d\  closer  to  zero  for  a  fixed  ij\ .  As  y\  approaches  its  minimum  value  (by  raising 
that  minimum  aspiration  level),  the  decision  maker  is  willing  to  decrease  y2  more  to 
increase  y\ . 

Increasing  L2  decreases  the  tradeoff.  The  minimum  level  of  y2  is  being  increased, 
forcing  d2  closer  to  zero  for  a  fixed  y2 .  As  y2  approaches  its  minimum  value  (by  raising 
that  minimum  aspiration  level),  the  decision  maker  is  willing  to  decrease  y2  less  to 
increase  y\ . 

This  tradeoff  analysis  conflicts  with  the  analysis  Kros  and  Mastrangelo  [29]  apply 
to  the  desirability  function.  Their  analysis  incorrectly  takes  the  partial  derivative 
of  the  desirability  function  with  respect  to  di  and  concludes  that  the  desirability 
function  implies  a  constant  trade-off  of  one  [29].  Their  analysis  considers  the  trade¬ 
off  with  respect  to  the  two  competing  desirabilities.  However,  the  trade-off  between 
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two  attributes  needs  to  be  calculated  with  respect  to  the  attributes  themselves  [31], 
not  their  desirability.  In  this  case,  that  is  with  respect  to  the  responses,  y\  and  y-2- 

3.2.2  Assessing  a  utility  function  over  the  desirability  function. 

When  uncertainty  is  present  in  a  decision  situation,  a  utility  function  is  required 
for  proper  assessment.  As  described  in  Chapter  II,  one  method  of  formulating  a 
multiattribute  utility  function  is  to  assign  a  single  attribute  utility  function  over 
a  multiattribute  value  function.  Assume  a  2-dimensional  deterministic  desirability 
function  (Equation  3.1).  Consider  the  case  of  constant  risk  aversion  over  desirability 
as  a  value  function.  Assume  the  following  utility  function 

U  =  1  -  e~^D,  (3.7) 

where  D  is  the  value  of  the  desirability  function  and  7  is  the  constant  risk  aversion 
with  respect  to  this  desirability.  Assume  the  decision  maker  is  risk  averse  with  respect 
to  desirability  (i.e.,  7  >  0).  Howard  [19]  notes  through  practical  experience  that  most 
decision  makers  are  risk  averse  in  their  attitude  for  risk  taking. 

Following  the  concepts  developed  by  Matheson  and  Abbas  [31],  the  risk  aversion 
function  for  each  attribute  is  expressed.  The  utility  function’s  risk  aversion  with 
respect  to  the  first  response,  yi,  is  examined  first.  The  risk  aversion  with  respect  to 
the  second  response  follows. 

The  risk  aversion  of  the  utility  function  with  respect  to  the  first  attribute,  y±,  is 

7£=7J%+t£-  (3-8) 
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Consider  each  term  in  Equation  3.8.  The  risk  aversion  for  the  utility  function  with 


respect  to  the  desirability  function  is  simply 


7  d  =  7- 


(3,9) 


The  partial  derivative  of  D  with  respect  to  ij\  is 


D'  = _ 

yi  2(Ti  -  Lx) 


\yi-Li 

II-l 

2 

1)2  ~  L2  ] 

i 

to-1 

1 

to 

1 - 

12 

2 


(3.10) 


The  contribution  of  the  desirability  function  to  the  risk  aversion  of  the  attribute 

yi  is 


i  _  n 

ryD  =  2 

iy  i 


(3,11) 


yi  -  L  i 

Substituting  these  expressions  into  Equation  3.8  provides  the  risk  aversion  ex¬ 
pressed  by  this  utility  function  with  respect  to  y\. 


1U  = 
IV 1 


7ri 


V1  2(yi  -  L,) 


r  2/1  -  li] 

rl 

2 

y  2  —  l2 

i 

8-i 

i>  —  l2_ 

1 2 
2 


+ 


i  _  n 
_ _ 2_ 

1-Ji  ~  Lx 


(3.12) 


From  Equation  3.12,  it  is  clear  that  the  risk  aversion  with  respect  to  y\  is  de¬ 
pendent  on  the  ranges  of  both  responses,  the  values  of  both  responses,  the  exponent 
assigned  to  each  response,  and  the  constant  risk  aversion  coefficient  in  the  utility  func¬ 
tion  over  desirability.  Based  on  the  assumptions  made  in  this  case,  every  segment 
of  this  expression  is  positive  with  the  exception  of  the  numerator  of  the  desirability 
function’s  contribution  to  risk  aversion,  1  —  y.  The  sign  of  this  contribution  depends 
on  the  exponent  rq  and  the  sign  switches  at  rq  =  2.  Examining  the  cases  of  rq  less 
than,  equal  to,  and  greater  than  two  provides  further  insight. 

For  rq  <  2,  each  component  of  Equation  3.12  is  positive;  therefore,  the  utility 
function  indicates  risk  aversion  with  respect  to  y\ .  That  being  said,  this  aversion  to 


31 


risk  varies  with  respect  to  changes  in  the  inputs  to  the  risk  aversion  function.  Table 
3  shows  how  risk  aversion  changes  with  respect  to  individual  variables  increasing, 
assuming  all  other  inputs  remain  constant. 

Table  3.  Risk  aversion  varies  with  respect  to  different  variables  increasing  when  n  <  2 

Variable  increasing  Effect  on 
(all  else  constant) _ 7^ 


7 

increases 

yi 

decreases 

V2 

increases 

n 

varies 

r2 

decreases 

L\ 

increases 

L-2 

decreases 

T\ 

decreases 

t2 

decreases 

As  7,  the  risk  aversion  coefficient  with  respect  to  desirability,  increases,  showing 
increased  risk  aversion  over  desirability,  the  risk  aversion  with  respect  to  an  individual 
attribute  increases. 

As  ij\  increases,  the  utility  function  displays  a  decreased  risk  aversion  to  y\ .  The 
closer  y\  is  to  its  target  value,  the  decision  maker  can  better  afford  taking  a  risk  in 

The  variables  associated  with  the  other  response,  r/2,  and  L-2,  affect  risk  prefer¬ 
ence  due  to  the  interaction  within  the  desirability  function.  As  mentioned  in  Chapter 
II  and  explicitly  indicated  by  7^ ,  the  risk  aversion  with  respect  to  a  single  attribute 
is  defined  in  part  by  the  form  of  the  value  function.  In  this  case,  the  desirability  func¬ 
tion  is  a  value  function.  The  risk  attitude  reaction  to  increases  in  y2  and  r2  match 
exactly  with  the  tradeoff  reactions  in  Table  2. 
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How  7^  varies  with  respect  to  r\  is  more  complicated.  Equation  3.13  shows  the 
partial  derivative  of  the  risk  aversion,  7^,  with  respect  to  r±. 


d_ 

dr\ 


1U 
iy  1 


1 

4(2/i  -  Li) 


2/i  -  jnl  2 

Ti-Lx 


2/2  ~  T2 

T2  —  L2 


r2 

2 


2  +  ri  In 


2/i  ~  £1 

T,  -L, 


-  2 
(3.13) 


This  partial  derivative  is  not  strictly  positive  or  negative.  The  sign  of  this  par¬ 
tial  derivative  depends  on  the  expression  within  the  parentheses.  If  d\  <  e~2,  then 
7F  <  0.  However,  the  converse  is  not  true. 

Figure  5  shows  isopreference  curves  for  various  values  of  7,  with  d\  on  the  hor¬ 
izontal  axis  and  d2  011  the  vertical  axis.  If  (o?i,  d2)  lies  on  an  isopreference  curve, 
=  ^  (di,d2)  lies  southwest  of  this  isopreference  curve,  ^7^  <  0  and  if 

(di,(i2)  lies  northeast  of  this  curve,  ^-7^  >  0. 


Figure  5.  Isopreference  curves  of  7  with  respect  to  (d-[ ,  d2)  that  make  ^-7^  =  0 


33 


The  decision  maker  must  be  aware  of  unintentional  consequences  of  adjusting 
r'i .  Figure  5  shows  how  increasing  rq  increases  risk  aversion  with  respect  to  y\  for 
a  particular  area  of  the  desirability  while  simultaneously  decreasing  risk  aversion 
with  respect  to  rq  for  its  complimentary  area.  As  7  decreases,  the  decision  maker  is 
moving  towards  risk  neutrality.  This  action  also  decreases  the  area  where  ^7^  >  0 
and  increases  the  area  where  7^-7^  <  0 

As  L\  increases,  risk  aversion  increases.  The  minimum  level  of  rq  is  being  in¬ 
creased,  forcing  d\  closer  to  zero  for  a  fixed  y\ .  As  rq  approaches  its  minimum  value 
(by  raising  that  minimum  aspiration  level)  the  utility  function  expresses  more  risk 
aversion  with  respect  to  rq . 

As  Ti  increases,  risk  aversion  decreases.  Since  a  fixed  value  of  rq  is  being  placed 
further  from  its  target,  the  utility  function  expresses  less  risk  aversion.  Increasing  the 
size  of  the  interval  [L 1,  Ti]  decreases  risk  aversion  with  respect  to  rq  while  decreasing 
the  size  of  the  interval  [Ti,  Ti]  increases  risk  aversion  with  respect  to  rq . 

As  T-2  increases,  risk  aversion  decreases.  Since  a  fixed  value  of  y2  is  being  placed 
further  from  its  target,  the  utility  function  expresses  less  risk  aversion. 

For  rq  =  2,  parts  of  Equation  3.12  cancel  out,  leaving  only  positive  expressions. 
It  follows  that  the  utility  function  expresses  risk  aversion  with  respect  to  rq .  This 
aversion  to  risk  varies  with  respect  to  changes  in  the  inputs  to  the  risk  aversion 
function.  Table  4  shows  how  risk  aversion  changes  with  respect  to  individual  variables 
increasing,  assuming  all  other  inputs  remain  constant. 

The  only  difference  between  Table  4  and  Table  3  (denoted  by  the  asterisk)  is  that 
the  response  rq  has  no  effect  on  the  risk  aversion  expressed  with  respect  to  y\ .  This 
is  due  to  the  fact  that  the  exponent  rq/2  =  1  and  yi  then  cancel  out  of  7^.  When 
rq  =  2,  the  desirability  function  is  said  to  be  risk-contribution  neutral  with  respect 
to  yi  since  7^  =  0  [31]. 
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Table  4.  Risk  aversion  varies  with  respect  to  different  variables  increasing  when  r1  =  2 


Variable  increasing  Effect  on 
(all  else  constant) _ 7^ 


7 

increases 

y\ 

no  effect* 

V2 

increases 

r2 

decreases 

L\ 

increases 

L-2 

decreases 

T\ 

decreases 

t2 

decreases 

For  r  1  >  2,  the  desirability  function’s  contribution  to  risk  aversion,  7^,  is  negative. 
The  utility  function  can  express  risk  seeking,  risk  neutral,  or  risk  aversion  with  respect 
to  y\ .  Table  5  shows  how  risk  aversion  changes  with  respect  to  individual  variables 
increasing,  assuming  all  other  inputs  remain  constant. 

Table  5.  Risk  aversion  varies  with  respect  to  different  variables  increasing  when  n  >  2 

Variable  increasing  Effect  on 
(all  else  constant)  7^ 


7 

increases 

yi 

increases* 

y2 

increases 

n 

varies 

r2 

decreases 

L\ 

varies* 

L-2 

decreases 

T\ 

decreases 

t2 

decreases 

When  comparing  the  results  shown  in  Table  5  with  the  results  shown  in  Table  3, 
it  is  seen  that  the  utility  function  displays  increasing  risk  aversion  with  respect  to 
yi  as  the  response,  y\ ,  increases.  Increasing  risk  aversion  is  not  how  most  decision 
makers  prefer  their  risk  aversion  to  be  modeled. 

Another  change  in  this  case  is  how  L\  affects  risk  aversion  with  respect  to  y\ . 
Equations  3.14  and  3.15  display  the  partial  derivative  of  7^  with  respect  to  L\. 
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d 

dLi 


1U 

iyi 


1 

2(2/1  -  L,y 


yi  -  Li 
T\  -  L 


112  ~  Li  I  2  /  _  r i(Ti  -  j/i)\ 

T2-L2J  V  2(Ti  -  Li)/ 


+  2  -  r1 


1 

2(2/i  -  Ti)2 


7'ri-D 


V  m-^)) 


+  2  -  rx 


(3.14) 

(3.15) 


This  partial  derivative  is  no  longer  monotonic.  Table  6  displays  the  sign  of  ^-7^ 
based  on  the  value  of  7 7.  When  ri  G  [2,  the  sign  °f  can  be  either  pos¬ 

itive  or  negative  depending  on  the  relationship  of  the  parameters  within  the  brackets 
of  Equation  3.15. 


Table  6.  Sign  of  -yU  baseci  on  ri 


Value  of 
ri 


Sign  of 

_d_~u 

fir,,  1  vi 


r  1  <  2 

2(Ti-Li) 
Ti—yi 
2  (Tx-lJ 
1  ^  T!-yi 


2  <  ri  < 


+ 

+/o/- 


The  decision  maker  must  be  aware  of  the  unintended  consequences  adjusting  the 
bounds  on  responses  has  on  risk  preference. 


3.2.3  Conjugate  Desirability  Function. 

The  exponential  utility  function  over  desirability  example  produced  a  case  where 
the  utility  function  over  value  was  from  a  different  family  of  utility  functions  than  the 
marginal  utility  functions  over  the  attributes.  In  many  cases,  a  decision  maker  may 
want  to  express  the  same  risk  preference  for  all  the  attributes  as  is  expressed  for  the 
given  value  function.  In  considering  this  case,  Matheson  and  Abbas  present  the  idea 
of  a  conjugate  value  function  [31]. 
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A  value  function  is  said  to  be  conjugate  to  a  utility  function  over  value  if  the 
marginal  utility  function  over  an  attribute  is  of  the  same  family  as  the  utility  function 
over  value  [31]. 

Consider  a  logarithmic  utility  function  over  desirability  as  shown  in  Equation  3.16. 


U  —  In  D  (3.16) 

The  risk  aversion  with  respect  to  desirability  is 

Id  =  1-  (3.17) 

The  partial  derivative  of  D  with  respect  to  y\,  D'  ,  and  the  contribution  of  the 
desirability  function  to  the  risk  aversion  of  y\ ,  7^  are  shown  in  Equations  3.10  and 
3.11  respectively.  The  risk  aversion  with  respect  to  ij\  is 


1U  = 
iy  1 


(3.18) 


Vi  -  L 1 

Matheson  and  Abbas  state  “the  product  value  function  is  conjugate  to  the  loga¬ 
rithmic  utility  function”  [31].  It  follows  that  the  desirability  function  is  conjugate  to 
the  logarithmic  utility  function. 


3.2.4  Desirability  function  as  a  utility  function. 

If  the  desirability  function  itself  is  assumed  to  be  a  utility  function,  it  follows  Abbas 
and  Howard’s  definition  of  an  attribute  dominance  utility  function  [4]  since  its  value 
is  zero  if  any  of  the  attributes  is  at  or  below  its  minimum  acceptable  value.  When 
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considering  the  desirability  function  in  this  manner,  it  is  denoted  as  Dd.  Equation 
3.19  shows  the  two-dimensional  case  when  L*  <  yl  <  Ti,i  =  1,2. 


Dd 


2/12/2 


(2/1, 2/2)  = 


\y1-L1~ 

ri 

2 

2/2 —  l2 

1 

_t2  —  L2_ 

(3.19) 


Equation  3.20  gives  the  marginal  utility  function  for  the  first  attribute. 


Dd  =  Dd 


2/1 


2/12/2 


{yi,T2)  = 


-1  Hi 

yi  -  L\ 1  2 


(3.20) 


T1-L1 

Equation  3.21  gives  the  conditional  utility  function  for  the  second  attribute  given 
a  certain  level  of  the  first  attribute. 


■^2/2 1 2/1  (2/2 1 2/i) 


Dt^(yi,y2) 


2/12/2' 


Dd 

2/1 


1/2 


(3.21) 


T2  —  ^2  _ 

Since  Equation  3.21  does  not  depend  on  the  value  of  7/1,  the  conditional  utility 
function  is  equal  to  the  marginal  utility  function  and  it  is  seen  that  the  desirability 
function  displays  utility  independence  of  the  two  attributes.  The  desirability  function 
can  then  be  expressed  as  in  Equation  3.22. 


Dim(yi,V2)  =  Ddyi(Vl)Ddm(y2)  (3.22) 

With  this  assumption  of  utility  independence,  the  analyst  can  simply  assess  the 
marginal  utility  functions  of  each  attribute  separately  and  then  multiply  them  to 
create  the  overall  desirability  function.  Equation  3.22  is  a  special  case  of  Equation 
2.6  which  depicts  the  general  case  of  a  utility  function  expressing  utility  independence. 

The  attribute  y\  is  utility  independent  of  7/2  if  the  conditional  preferences  of  y\ 
given  a  certain  value  of  y2  do  not  depend  on  the  value  of  y2  [22],  If  the  attributes 
are  not  utility  independent,  the  analyst  should  not  use  the  desirability  function  as  a 
utility  function. 
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Equation  3.23  gives  the  risk  aversion  of  the  marginal  utility  function  of  the  first 


attribute. 


7j/i 


i  _  tk 

1  2 


(3.23) 


Vi  -  Lx 

Kros  and  Mastrangelo  present  erroneous  conclusions  regarding  the  risk  aversion 
of  the  desirability  function  [29].  For  r*  =  2,  the  desirability  function  reflects  risk 
neutrality  with  respect  to  y%.  For  rt  <  2  the  desirability  function  reflects  decreasingly 
risk  averse  behavior.  For  r*  >  2,  the  desirability  function  reflects  decreasingly  risk 
seeking  behavior.  This  breakpoint  in  modeling  risk  preference  is  dependent  on  the 
number  of  objectives.  When  generalized  to  n-dimensions,  the  desirability  function 
reflects  risk  neutrality  when  rt  =  n.  The  desirability  function  reflects  risk  averse  and 
risk  seeking  behavior  when  r*  <  n  and  rt  >  n  respectively. 

The  mixed  partial  derivative  of  the  desirability  function  is  shown  in  Equation  3.24. 


dyidy2 


rxr2 

4(2/i  -  Lx)(y2 


r 2/1  -  Lx] 

ri 

2 

\y2-  l2] 

1 

to 

1 

t-i 

to 

1 - 

(3.24) 


This  mixed  partial  derivative  is  always  positive  over  the  range  <  y^  <  T^i  — 
1,  2.  Therefore,  the  desirability  function  displays  multivariate  risk  seeking  behavior 
[39].  This  type  of  behavior  makes  sense  in  an  attribute  dominance  utility  function, 
since  the  overall  utility  is  zero  if  any  one  of  the  attributes  is  at  its  minimum  level. 

The  fact  that  the  desirability  function  sets  the  overall  desirability  to  zero  if  any 
of  its  attributes  is  at  or  below  its  minimum  values,  regardless  of  the  values  of  the  re¬ 
maining  attributes  merits  discussion.  Consider  the  following  three  attribute  relation. 


D(Li  T  e,  L2  T  e,  L3  +  e)  >  D(L  1  —  e,  T2,  X3)  —  0, 


(3.25) 
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where  e  >  0  is  small.  Consider  the  right  hand  side  of  Equation  3.25.  Even  with  the 
second  two  attributes  at  their  target  values,  since  the  first  attribute  is  slightly  below 
its  minimum  value,  its  desirability  is  zero.  Consider  the  left  hand  side  of  Equation 
3.25.  With  all  three  attributes  slightly  above  their  minimum  values,  the  desirability 
is  a  small  positive  number  and  is  therefore  greater  than  the  right  hand  side.  Care 
must  be  taken  to  ensure  a  valid  minimum  value  is  decided  upon  before  applying  the 
desirability  function.  An  elevated  minimum  value  could  reject  a  potentially  acceptable 
decision  point.  Moreover,  changing  the  minimum  values  of  the  responses  also  affects 
the  trade-off  and  risk  aversion  functions.  In  decision  analysis,  a  value  (or  utility) 
function  is  stipulated  over  any  valid  domain  of  interest  in  the  attempt  to  avoid  such 
errors  (i.e.,  L,  =  y 9  and  Tj:  =  y *,  Vi). 

3.3  A  Decision  Analysis  Perspective  on  Robust  Optimization 

A  set  of  regression  equations  is  developed  from  an  experimental  design  over  the 
decision  space.  Once  the  decision  maker  selects  the  proper  user-defined  parameters, 
the  maximum  desirability  is  calculated  and  its  location  found  within  the  feasible 
region. 

Since  the  desirability  function  is  not  differentiable  throughout  its  range,  a  non- 
gradient  optimization  algorithm  must  be  used.  The  Hooke  and  Jeeves  Direct  Search 
algorithm  [17,  20,  8]  is  implemented  in  Excel  using  Visual  Basic  for  Applications 
(VBA).  Since  the  starting  point  is  critical  when  using  such  an  algorithm  over  such 
un-smooth  surfaces,  multiple  starting  points  are  chosen  throughout  the  feasible  region. 
This  is  accomplished  by  taking  a  random  sampling  of  the  feasible  region  or  by  dividing 
the  feasible  region  into  a  grid  of  equally  spaced  vertices  along  each  dimension  of  the 
decision  space.  The  code  iterates  through  each  starting  point  for  the  direct  search 
algorithm.  The  code  outputs  the  location  and  desirability  of  all  local  maxima.  To 
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confirm  the  results  of  the  direct  search  algorithm,  the  local  maxima  is  also  calculated 
using  the  differential  evolution  algorithm  within  Matlab  [38] .  Once  the  deterministic 
local  maxima  are  calculated,  the  Direct  Search  algorithm  is  re-run  calculating  the 
desirability  function  in  a  Monte  Carlo  simulation. 

Clemen  and  Reilly  discuss  using  regression  equations  for  decision  analysis.  These 
equations  estimate  the  conditional  expected  value  of  a  response  given  a  set  of  input 
variables.  The  difference  between  the  observed  value  and  the  predicted  value  of  the 
response  is  the  residual.  This  set  of  residuals  can  be  used  to  estimate  the  continuous 
distribution  function  (CDF)  of  the  random  error  term,  e,  as  described  in  Equation 
2.18  in  Chapter  11  [11],  This  CDF  is  approximated  with  a  Beta  distribution  using 
the  residuals  as  the  fractile  midpoints  in  Matlab’s  fminsearch  function  as  described 
in  Abbas  et  al  [3]  and  reviewed  in  Section  2.2.4. 

The  correlation  of  the  residuals,  i?,  is  calculated  and  then  the  random  samples  for 
the  Monte  Carlo  simulation  is  calculated  using  the  multivariate  normal  copula  [10]. 

Matlab’s  mvnrnd  function  generates  a  matrix  of  random  vectors  from  a  multivari¬ 
ate  standard  normal  distribution  with  correlation  R.  Equation  3.26  displays  how  the 
vector  of  random  samples  (ei, . . . ,  en)  is  calculated, 

£„)  =  (FfWs/Oli ....  F-'lfcGfa)]),  (3.26) 

where  <f>  is  the  standard  normal  CDF  and  F)_1,  i  =  1, . . .  ,  n  is  the  i-th  inverse  Beta 
marginal  CDF  estimated  from  the  residuals. 

Using  each  local  maximum  as  the  starting  point,  the  solution  to 

max  (~D(y{1)  +  (3-27) 

y(1)  ~i\n  J 
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is  calculated  where  n  is  the  number  of  random  samples  used  in  the  Monte  Carlo 
simulation,  e*  is  i-th  vector  random  samples  and  t/1)  is  the  vector  of  responses  found 
with  the  highest  expected  desirability.  The  global  maximum  is  the  robust  solution 
point  for  the  system.  Sensitivity  analysis  indicates  the  robustness  of  the  solution. 
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IV.  Wire  Bonding  Experiment  Findings 


4.1  Multi-response  optimization  of  semiconductor  manufacturing  process 

Del  Castillo  et  al.  [9]  introduce  an  experiment  to  illustrate  a  multi-response  op¬ 
timization  problem  using  the  desirability  function.  The  experiment  is  executed  to 
optimize  a  wire  bonding  process  in  the  semiconductor  industry.  The  experiment  has 
three  input  factors  and  six  responses.  Table  7  contains  the  three  factors  and  their  low 
and  high  values  used  in  the  experiment.  Table  8  contains  the  response  descriptions 
and  their  acceptable  ranges  and  target  values. 

Table  7.  Factors  and  levels  for  example  experiment  [9] 


Factor 

Factor 

Name 

Levels 
Low  High 

A 

Flow  Rate 

40 

120 

B 

Flow  Temp 

200 

450 

C 

Block  Temp 

150 

350 

Table  8.  Response  descriptions  and  ranges  for  multiple  response  example  [9] 


Response  -  description 

Min 

Max 

Target 

yi 

=  maximum  temperature  at  Position  A 

185 

195 

190 

V2 

=  beginning  bond  temperature  at  position  A 

170 

195 

185 

V3 

=  finish  bond  temperature  at  position  A 

170 

195 

185 

Va 

=  maximum  temperature  at  position  B 

185 

195 

190 

V5 

=  beginning  bond  temperature  at  position  B 

170 

195 

185 

ye 

=  finish  bond  temperature  at  position  B 

170 

195 

185 

Excel  Analysis  ToolPak  is  used  to  run  an  ordinary  least  squares  (OLS)  multiple 
linear  regression  on  the  data  del  Castillo  et  al.  provide  from  the  experiment  to 
estimate  the  response  equations  [9]. 
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yx  =  -18.404  +  0.567a;2  +  0.530x3  -  0.002x2x3  (4.1) 

y2  =  37.463  +  0.150x4  +  0.173x2  +  0.141x3  (4.2) 

y3  =  33.413  +  0.166x4  +  0.128x2  +  0.204x3  (4.3) 

)/4  =  45.616  +  0.767x4  +  0.065x2  +  0.079x3  —  O.OO8X4  +  0.002xix2  (4.4) 

y5  =  51.443  +  0.230xi  +  0.046x2  +  0.165x3  -  0.004x?  +  0.001xix2  (4.5) 

ye  =  36.066  +  0.239xi  +  0.049x2  +  0.274x3  —  0.003x^  +  0.001xix2  (4.6) 


4.2  Deterministic  optimum  setting 

The  two-sided  desirability  function  from  Equation  2.23  is  used  to  find  the  optimum 
operating  environment  within  the  experimental  region  described  in  Table  7.  Using 
the  Min.  Max,  and  Target  values  from  Table  8  as  the  values  of  Li,  Ui,  T{,  i  =  1, . . . ,  6 
and  Equations  4.1  through  4.6,  the  Hooke-Jeeves  (HJ)  algorithm  finds  the  maximum 
desirability  and  its  operating  solution  for  various  risk  preferences.  Table  9  contains 
the  exponent  parameters  used  to  express  various  risk  attitudes. 

Table  9.  Exponent  parameters  used  to  express  various  risk  attitudes 

Risk  Attitude 
Averse  Neutral  Seeking 

Si,ti  3  6  9 

In  each  of  the  risk  attitude  cases  described  in  Table  9,  the  same  two  local  optimum 
solutions  are  found.  Adjusting  the  exponents  s*  and  U  {i  =  1, . . . ,  6)  such  that  st  =  ti 
(■ i  =  1, . . . ,  6)  will  not  change  the  optimum  solution  because  the  trade-off  functions, 
t(yi,yj),  i  ^  j  does  not  change. 
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Table  10  displays  the  two  local  optimum  solutions  and  their  predicted  responses. 


The  desirability  values  shown  are  from  the  risk  neutral  case  where  s*  =  t,  =  6 
(i  —  1, . . .  ,6).  The  first  column  contains  the  global  optimum  solution. 

Table  10.  Local  deterministic  operating  solutions 


Local  Optimum  Points 


Xl 

x2 

X3 

120.000 

436.168 

320.297 

82.192 

450.000 

330.453 

yi 

186.391 

185.939 

m 

176.162 

174.318 

ys 

174.378 

171.955 

y 4 

193.277 

192.861 

£5 

173.139 

172.910 

ye 

184.364 

185.000 

D 

0.0023 

0.0006 

These  two  local  optimum  solutions  represent  the  two  regions,  A  and  B,  where 
D  >  0  in  the  experimental  region.  Figure  6  represents  Region  A  by  displaying 
desirability  as  Block  Temperature  and  Flow  Rate  vary  with  Flow  Temperature  fixed 
at  X2  =  436.17.  Figure  7  represents  Region  B  by  displaying  desirability  as  Block 
Temperature  and  Flow  Rate  vary  with  Flow  Temperature  fixed  at  X2  =  450. 

The  figures  displaying  Regions  A  and  B  with  Flow  Temperature  and  Block  Tem¬ 
perature  held  constant  are  presented  in  Appendix  C.  From  all  three  dimensions  of  x, 
Region  A  appears  larger  than  Region  B.  Region  A  also  contains  the  global  optimum 
solution. 

Both  del  Castillo  et  al  [9]  and  Kros  and  Mastrangelo  [29]  display  results  regarding 
a  global  maximum  for  this  experimental  region  using  s*  =  L  =  1,  i  —  1, . . . ,  6.  Table 
11  compares  the  maximum  solution  displayed  in  both  of  these  articles. 

Del  Castillo  et  al.’s  solution  is  displayed  in  the  first  column  of  Table  11.  Kros 
and  Mastrangelo’s  solution  is  in  the  second  column;  it  matches  del  Castillo  et  al.’s 
solution  closely.  The  third  column  is  the  solution  found  using  the  regression  equations 


45 


Figure  6.  Value  of  D  as  Block  Temp  and  Flow  Rate  vary  (Flow  Temp  =  436.17) 


Figure  7.  Value  of  D  as  Block  Temp  and  Flow  Rate  vary  (Flow  Temp  =  450) 
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Table  11.  Comparing  global  maximum  found  for  the  del  Castillo  example[9,  29] 


DC 

KM 

Self 

Xl 

84.312 

84.14 

84.113 

X2 

450 

450 

450 

X3 

329.73 

329.66 

329.902 

Vl 

186.1 

186.06 

186.02 

Vi 

174.5 

174.49 

173.92 

Vs 

172.1 

172.02 

172.06 

V4 

192.6 

192.63 

192.62 

Vs 

173.1 

173.04 

173.07 

y& 

185 

184.95 

185.00 

D 

0.3076 

0.3058 

0.2989 

stated  by  del  Castillo  et  al.  [9]  in  the  H  J  algorithm.  Interestingly,  this  set  of  equations 
produces  a  global  maximium  in  Region  B  instead  of  region  A  as  when  using  Equations 
4.1  through  4.6. 

4.3  Monte  Carlo  Simulation 

The  residuals  based  on  the  above  model  are  used  to  estimate  the  marginal  dis¬ 
tribution  functions  for  the  six  responses.  Appendix  B  contains  the  residuals  for  this 
regression  model.  The  lower  bound  A  for  each  response’s  Beta  distribution  is  chosen 
by  rounding  its  lowest  residual  down  to  the  next  integer  value.  The  upper  bound  B 
for  each  response  is  chosen  by  rounding  its  highest  residual  up  to  the  next  integer 
value.  Using  these  bounds,  the  a  and  /3  parameters  are  estimated  using  Matlab’s 
fminsearch  function.  Table  12  contains  the  estimated  parameters  of  the  six  marginal 
Beta  distribution  functions. 


Table  12.  Estimated  parameters  for  the  marginal  Beta  distribution  functions 


Residual 

Vi 

Residual 

Vi 

Residual 

Vs 

Residual 

V4 

Residual 

V5 

Residual 

V6 

a 

0.9980 

0.9047 

1.3739 

0.8615 

1.1511 

1.1428 

(3 

1.1914 

0.9218 

1.3411 

0.9916 

1.5732 

1.2991 

A 

-15 

-11 

-16 

-9 

-5 

-5 

B 

31 

21 

30 

18 

11 

10 
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The  correlation  matrix  R  is  calculated  from  the  residuals  as  shown  in  Table  13. 


Table  13.  Correlation  matrix  of  the  residuals 


1 

0.7672 

0.6728 

0.7672 

1 

0.8351 

0.6728 

0.8351 

1 

0.4184 

0.6260 

0.5084 

0.5533 

0.8313 

0.6988 

0.3700 

0.6369 

0.5849 

0.4184 

0.5533 

0.3700 

0.6260 

0.8313 

0.6369 

0.5084 

0.6988 

0.5849 

1 

0.8838 

0.9232 

0.8838 

1 

0.9122 

0.9232 

0.9122 

1 

Using  Matlab’s  mvnrnd  function  and  the  normal  multivariate  copula  method,  a 
set  of  10,000  sample  residuals  are  constructed  for  the  Monte  Carlo  simulation.  To 
confirm  this  random  sample  is  from  the  intended  distribution,  the  Beta  distribution 
parameter  estimates  and  correlation  matrix  of  the  sample  is  calculated  for  comparison 
with  the  residual  data  estimates.  Table  14  contains  the  Beta  parameters  fit  to  the 
sample. 


Table  14.  Beta  distribution  parameters  fit  to  the  Monte  Carlo  samples 


Residual 

Residual 

Residual 

Residual 

Residual 

Residual 

Vi 

V2 

Vs 

Z/4 

Z/5 

Ve 

a 

0.9688 

0.8878 

1.3584 

0.8569 

1.1485 

1.1336 

1.1618 

0.9125 

1.337 

0.9789 

1.571 

1.2833 

Table  15  contains  the  correlation  matrix  of  the  random  sample.  Both  the  param¬ 
eters  from  the  six  marginal  distributions  and  the  correlation  matrix  from  the  random 
sample  resemble  those  calculated  from  the  residuals. 

Table  15.  Correlation  matrix  for  the  Monte  Carlo  samples 


1 

0.7545 

0.6592 

0.4121 

0.5452 

0.3660 

0.7545 

1 

0.8211 

0.6092 

0.8169 

0.6224 

0.6592 

0.8211 

1 

0.4859 

0.6791 

0.5665 

0.4121 

0.6092 

0.4859 

1 

0.8712 

0.9127 

0.5452 

0.8169 

0.6791 

0.8712 

1 

0.9056 

0.3660 

0.6224 

0.5665 

0.9127 

0.9056 

1 
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The  Monte  Carlo  simulation  is  run  using  the  random  sample.  First,  the  desirability 
function  is  treated  as  a  utility  function.  The  HJ  algorithm  is  started  at  both  of  the 
local  maxima  displayed  in  Table  10.  Risk  aversion  is  modeled  as  described  in  Table 
9.  Table  16  displays  the  robust  optimal  solutions  when  s*  =  U  =  3, 6,  9  (i  —  1 . . . ,  6). 

Table  16.  Robust  optimal  solutions  varying  by  risk  aversion 


^•£5  ^ i 

3 

6 

9 

Xl 

120 

120 

120 

x2 

407.190 

429.415 

431.174 

x3 

349.934 

339.771 

337.703 

E(D) 

0.0212 

0.0037 

0.0009 

The  robust  optimal  solution  moves  slightly  as  the  exponents  vary,  but  it  remains 
near  Region  A.  The  value  of  E(D)  decreases  as  the  exponent  increases  due  to  the 
decreasing  effect  the  exponent  has  on  d,;.  It  appears,  in  this  case,  adjusting  the 
exponents  in  the  desirability  function  does  not  affect  which  area  of  the  operating 
region  contains  the  robust  optimal  solution  when  the  desirability  function  is  used  as 
a  utility  function.  This  is  most  likely  due  to  Region  A  containing  the  highest  peak 
desirability  and  also  being  the  larger  region  where  D  >  0. 

The  robust  optimal  solution  is  also  calculated  using  the  desirability  function  as  a 
value  function  and  assessing  an  exponential  utility  function  over  it. 

U  =  1  -  e~^D  (4.7) 

The  exponents,  s*,  tl  ( i  =  1, ...  ,6)  vary  as  described  in  Table  9  in  addition  to 
varying  the  risk  aversion  coefficient,  7  =  0.2, 1,  5.  Table  17  contains  the  set  of  robust 
optimal  solutions  found  in  this  case. 

The  robust  optimal  solution  moves  slightly  as  the  exponents  and  7  vary,  but  it 
remains  near  Region  A.  The  value  of  E(U)  decreases  as  the  exponent  increases  due 
to  the  decreasing  effect  the  exponent  has  on  dj.  The  value  of  E(U )  increases  as  7 
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Table  17.  Robust  optimal  solutions  varying  7,  s,, 


7 

t i 

3 

6 

9 

0.2 

Xi 

120 

120 

120 

x2 

407.176 

429.413 

431.175 

x3 

349.990 

339.778 

337.703 

E(U) 

0.0042 

0.0007 

0.0002 

1 

Xi 

120 

120 

120 

x2 

407.192 

429.348 

431.329 

x3 

350 

339.854 

337.769 

E(U) 

0.0196 

0.0035 

0.0009 

5 

Xl 

120 

120 

120 

x2 

407.227 

429.018 

431.416 

X3 

350 

340.449 

337.873 

E(U) 

0.0739 

0.0156 

0.0043 

increases  due  to  the  increasing  effect  it  has  on  U.  It  appears,  in  this  case,  adjusting  the 
exponents  or  7  does  not  affect  which  area  of  the  operating  region  contains  the  robust 
optimal  solution  when  an  exponential  utility  function  is  assessed  over  the  desirability 
function.  This  is  most  likely  due  to  Region  A  containing  the  highest  peak  desirability 
and  also  being  the  larger  region  where  D  >  0. 

The  desirabilities  calculated  by  adding  the  10,000  sample  residuals  to  the  various 
optimum  solutions  found  provide  insight  into  the  robustness  of  the  solutions.  Consider 
the  case  when  s,  =  ti  —  3  (i  —  1, . . . ,  6).  Table  18  contains  the  average  and  maximum 
desirabilities  calculated  when  using  the  deterministic  solution  shown  in  Table  10  and 
the  robust  optimal  solutions  shown  in  Table  17. 

Table  18.  Maximum  and  average  desirability  for  deterministic  optimum  and  optimums 

at  7  =  0.2, 1,  5  when  st  =7  =  3 

Deterministic  7  =  0.2  7=1  7  =  5 

E(D)  0.0109  0.0212  0.0212  0.0212 

Max  D  0.5129  0.4112  0.4109  0.4104 


The  deterministic  solution  produces  a  higher  maximum  desirability  when  adding 
the  set  of  10,000  random  residuals  to  it.  However  it  produces  a  lower  average  de- 
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sirability.  The  robust  solutions  found  at  7  =  0.2, 1,  5  produce  different  maximum 
desirabilities,  however  the  average  desirability  across  this  set  is  the  same. 

Figure  8  displays  the  distribution  of  desirabilities  across  the  10,000  samples.  The 
deterministic  optimum  solution  is  compared  to  the  three  robust  solutions  found  with 
7  =  0.2, 1,5. 


■  Deterministic 

■  y  =  0.2 
■Y  =  l 

■  y  =  S 


Figure  8.  Compare  the  distribution  of  D  between  different  values  of  7  and  the  deter¬ 
ministic  optimum  when  s,;  =  f,  =  3) 

The  horizontal  axis  shows  desirability.  The  vertical  axis  shows  the  number  of  sam¬ 
ples  with  that  particular  desirability  level  on  a  logarithmic  scale.  The  distributions  of 
the  three  robust  solutions  are  very  similar.  However,  the  deterministic  solution  tends 
to  have  fewer  samples  with  higher  desirabilities  than  the  robust  solutions  have.  In 
this  case,  a  risk  averse  or  risk  neutral  decision  maker  would  prefer  one  of  the  robust 
solutions.  A  sufficiently  risk  seeking  decision  maker  would  prefer  the  deterministic 
solution. 
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Consider  the  exponential  utility  function 


U  =  e~lD  (4.8) 

where  7  <  0.  This  utility  function  describes  risk  seeking  behavior  with  respect  to 
desirability  and  7^  =  7.  When  7  <  —16.5,  the  deterministic  solution  is  preferred  to 
the  three  robust  solutions.  This  extreme  risk  seeking  attitude  is  rarely  modeled  in 
practice. 

Consider  the  case  when  Si  =  U  =  6  (i  =  1, . . . ,  6).  Table  19  contains  the  average 
and  maximum  desirabilities  calculated  when  using  the  deterministic  solution  shown 
in  Table  10  and  the  robust  optimal  solutions  shown  in  Table  17. 

Table  19.  Maximum  and  average  desirability  for  deterministic  optimum  and  optimums 

at  7  =  0.2, 1,  5  when  Sj  =  7  =  6 

Deterministic  7  =  0.2  7=1  7  =  5 

E(D)  0.0018  0.0037  0.0037  0.0037 

Max  D  0.2631  0.2873  0.2884  0.2893 

The  deterministic  solution  produces  a  lower  maximum  desirability  and  lower  av¬ 
erage  desirability  when  adding  the  set  of  10,000  random  residuals  to  it.  The  robust 
solutions  found  at  7  =  0.2, 1,  5  produce  different  maximum  desirabilities,  however  the 
average  desirability  across  this  set  is  the  same. 

Figure  9  displays  the  distribution  of  desirabilities  across  the  10,000  samples.  The 
deterministic  optimum  solution  is  compared  to  the  three  robust  solutions  found  with 
7  =  0.2, 1,5. 

The  horizontal  axis  shows  desirability.  The  vertical  axis  shows  number  of  sam¬ 
ples  with  that  particular  desirability  level  on  a  logarithmic  scale.  The  distributions 
of  the  three  robust  solutions  are  very  similar.  However,  the  deterministic  solution 
tends  to  have  fewer  samples  with  higher  desirabilities  than  the  robust  solutions  have. 
In  this  case,  since  the  robust  solutions  had  the  highest  maximum  desirability  and 
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D 


■  Deterministic 

■  y  =  0.2 

■  Y=1 

■y=5 


Figure  9.  Compare  the  distribution  of  D  between  different  values  of  7  and  the  deter¬ 
ministic  optimum  when  s,;  =  f,  =  6) 

highest  average  desirability,  a  decision  maker  would  prefer  a  robust  solution  over  the 
deterministic  solution  regardless  of  risk  preference. 

Consider  the  case  when  s*  =  i*  =  9  (i  —  1, . . . ,  6).  Table  20  contains  the  average 
and  maximum  desirabilities  calculated  when  using  the  deterministic  solution  shown 
in  Table  10  and  the  robust  optimal  solutions  shown  in  Table  17. 

Table  20.  Maximum  and  average  desirability  for  deterministic  optimum  and  optimums 

at  7  =  0.2, 1,  5  when  Sj  =  f  j  =  9 

Deterministic  7  —  0.2  7=1  7  =  5 

E(D)  0.0004  0.0009  0.0009  0.0009 

Max  D  0.1350  0.1488  0.1462  0.1434 

The  deterministic  solution  produces  a  lower  maximum  desirability  and  lower  av¬ 
erage  desirability  when  adding  the  set  of  10,000  random  residuals  to  it.  The  robust 
solutions  found  at  7  =  0.2, 1,  5  produce  different  maximum  desirabilities,  however  the 
average  desirability  across  this  set  is  the  same. 
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Figure  10  displays  the  distribution  of  desirabilities  across  the  10,000  samples.  The 
deterministic  optimum  solution  is  compared  to  the  three  robust  solutions  found  with 
7  =  0.2, 1,5. 


0  0.01  0.02  0.03  0.04  0.05  0.06  0.07  0.08  0.09  >0.1 

D 


Figure  10.  Compare  the  distribution  of  D  between  different  values  of  7  and  the  deter¬ 
ministic  optimum  when  s*  =  U  =  9) 

The  horizontal  axis  shows  desirability.  The  vertical  axis  shows  number  of  sam¬ 
ples  with  that  particular  desirability  level  on  a  logarithmic  scale.  The  distributions 
of  the  three  robust  solutions  are  very  similar.  However,  the  deterministic  solution 
tends  to  have  fewer  samples  with  higher  desirabilities  than  the  robust  solutions  have. 
In  this  case,  since  the  robust  solutions  had  the  highest  maximum  desirability  and 
highest  average  desirability,  a  decision  maker  would  prefer  a  robust  solution  over  the 
deterministic  solution  regardless  of  risk  preference. 
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V.  Robust  Stability  Operations  Policy  Findings 


5.1  National  Operational  Environment  Model  Experiment 

To  illustrate  the  decision  analysis  approach  to  robust  optimization,  a  stability 
operations  policy  optimization  problem  is  investigated  within  the  Air  Force  Research 
Laboratory’s  (AFRL)  National  Operational  Environment  Model  (NOEM). 

An  experiment  is  designed  to  investigate  how  various  diplomatic,  informational, 
military  and  economic  (DIME)  instruments  of  national  power  affect  the  Democratic 
Republic  of  Congo’s  (DRC)  political,  military,  economic,  social,  infrastructure,  and 
information  (PMESII)  systems. 

This  experiment  investigates  the  effect  14  DIME  factors  have  on  two  PMESII 
responses.  Table  21  lists  the  factors  with  brief  descriptions.  Table  22  lists  the  units 
and  minimum  and  maximum  values  for  the  experimental  factors. 

National  debt  and  total  number  of  activists  are  the  two  indicators  of  the  DRC’s 
PMESII  systems.  Minimizing  both  responses  is  preferred  and  is  indicative  of  a  more 
stable  government. 

Each  design  point  is  run  for  three  simulated  years  (1095  days)  with  two  replica¬ 
tions.  A  list  of  random  seed  generators  is  generated  in  Excel  and  each  design  point 
is  assigned  a  seed  from  this  list.  NOEM  outputs  debt  and  activist  data  for  every 
simulated  day.  This  experiment  considers  the  arithmetic  mean  of  the  last  30  days  of 
the  simulation’s  debt  and  activist  output  as  the  two  responses  for  debt  and  activists 
respectively. 
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Table  21.  NOEM  DRC  experiment  factors  and  descriptions 


Factor 

Diplomatic 

Stimulus  maximum 

Stimulus  Spending  % 
Government  Corruption 


Military 

Initial  Police  Forces 
Police  Forces  Goal 
Jail  Term 

Mean  Adjudication  Processing  Time 

Adjudication  Rate 

Economic 

Tax  Rate 

Interest  Rate 

Long  Term  Government  Employee 
Share 

Government  Wages 
Infrastructure  Spending  % 

Services  Spending  % 


Description 


Maximum  amount  of  money  allocated 
daily  to  government  stimulus 
Percentage  of  government  funding  ear¬ 
marked  for  stimulus 

Proportion  of  government  income  not 
available  for  use 


Police  forces  at  the  start  of  the  simulation 
Long  term  goal  for  police  forces 
Mean  jail  term  for  arrested  activist 
Mean  time  spend  in  adjudication  process 
Average  rate  of  adjudication  process 


Income  and  production  tax  rate 
Government  debt  interest  rate 
Long  term  proportion  of  workers  em¬ 
ployed  by  the  government 
Mean  annual  wage  paid  to  government 
employees 

Percentage  of  government  funding  ear¬ 
marked  for  infrastructure 
Percentage  of  government  funding  ear¬ 
marked  for  providing  services 
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Table  22.  NOEM  DRC  experiment  factors,  units,  minimum  values,  maximum  values 


Factor 

Units 

Min 

Max 

Diplomatic 

Stimulus  maximum 

$ 

0 

6,000,000 

Stimulus  Spending  % 

% 

0 

100 

Government  Corruption 

n/a 

0.05 

0.15 

Military 

Initial  Police  Forces 

personnel 

52,500 

137,500 

Police  Forces  Goal 

personnel 

112,500 

187,500 

Jail  Term 

days 

3 

100 

Mean  Adjudication  Processing  Time 

day 

0.1 

0.9 

Adjudication  Rate 

per  day 

0.01 

0.13 

Economic 

Tax  Rate 

n/a 

0.03 

0.5 

Interest  Rate 

n/a 

0.0225 

0.0675 

Long  Term  Government  Employee 

n/a 

0.001 

0.05 

Share 

Government  Wages 

$ 

1400 

4200 

Infrastructure  Spending  % 

% 

0 

100 

Services  Spending  % 

% 

0 

100 

The  experiment  executes  a  D-optimal  design.  D-optimality  focuses  on  good  model 
coefficient  estimation.  It  does  this  by  choosing  design  points  so  that  the  determinant 
of  the  moment  matrix  M  is  maximized  [32], 


\M\  = 


\X'X\ 

Np 


(5.1) 


where  X  is  the  design  matrix,  N  is  the  number  of  experiment  runs  and  p  is  the 
number  of  parameters  [32],  An  experimental  design  consisting  of  139  design  points 
with  two  replicates  each  is  chosen  to  create  a  full  quadratic  model  with  cubic  terms. 

A  deterministic  value  function,  V,  is  formulated  to  describe  the  preferred  rela¬ 
tionship  between  the  two  responses. 


V  =  18.164  -  5.75  x  10~10r/i  -  1.01  x  10 ~4y2,  (5.2) 
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where  tj\  is  debt  in  dollars,  and  y2  is  the  number  of  activists. 


V' 

t(y2,yi)  =  T^  =  176,439.50  (5.3) 

vyi 

The  value  function  V  is  describing  a  tradeoff  where  the  decision  maker  would  be 
willing  to  increase  the  debt  by  $176,439.50  in  order  to  reduce  activists  by  one. 

This  particular  form  of  a  value  function  assumes  a  constant  trade-off  between  the 
two  responses  throughout  the  entire  response  space.  Consider  two  examples,  one  with 
$3  billion  of  debt  and  500  activists,  and  a  second  with  $3  billion  of  debt  and  50,000 
activists.  It  is  unlikely  a  decision  maker  would  have  the  same  tradeoff  between  debt 
and  number  of  activists  in  these  two  cases.  Given  $3  billion  of  debt,  a  decision  maker 
would  more  likely  have  a  higher  tradeoff  when  faced  with  50,000  activists  as  opposed 
to  when  faced  with  500  activists.  Moreover,  given  a  particular  number  of  activists,  a 
decision  maker  would  more  likely  have  a  decreasing  tradeoff  as  debt  increases. 

Consider  an  exponential  utility  function  assessed  over  V 

r 

1  +  e~^v,  7  <  0 

U  =  V,  7  =  0.  (5.4) 

1  -  e~^v,  7  >  0 

After  assessing  a  utility  function  over  value,  the  risk  aversion  with  respect  to  the 
two  attributes  can  be  calculated.  The  objective  in  this  experiment  is  to  maximize  V 
which  minimizes  y\  and  y2 ■  Since  this  is  a  decreasing  value  function,  the  risk  aversion 
functions  with  respect  to  these  attributes  change  some  of  the  signs  in  Equations  2.11 
and  2.10. 
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(5.5) 


V  = 

I  Vi 


1U  = 
'Vi 


V" 

i  =  1,2 


(5.6) 


This  value  function  is  additive,  so  its  contribution  to  risk  aversion  is  zero. 


ll  =  0,  *  =  1,2 


'Vi 


(5.7) 


The  risk  aversion  of  the  utility  function  with  respect  to  the  two  attributes  are 


7^  =  5>75  x  10-107  and  (5.8) 

lUm  =  1.01  x  KT47.  (5.9) 

In  cases  where  constant  risk  aversion  is  indicated  by  the  utility  function,  one  must 
be  cognizant  of  saturation  effect.  The  utility  function,  U  =  1  —  e~lV ,  equals  one  as 
V  approaches  positive  utility.  When  V  is  greater  than  «  I/7,  the  utility  function  is 
nearly  a  horizontal  line.  This  saturation  effect  improperly  models  risk  preference  at 

these  higher  levels  of  V.  This  limitation  of  the  exponential  utility  function  must  be 

considered  when  selecting  reasonable  values  of  7. 

5.2  Experiment  Results 

After  the  experiment  is  run,  the  data  is  analyzed  in  Design- Expert.  Ordinary 
least  squares  (OLS)  regression  is  applied  and  two  regression  equations  to  predict  y\ 
and  i/2  are  developed. 
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The  model  for  y\  did  not  pass  the  lack-of-fit  test.  However  the  R-Squared,  adjusted 
R- Squared  and  predicted  R-Squared  measures  are  all  high  so  this  model  is  accepted. 
After  a  natural  log  transform  is  applied  to  the  y2  response  this  model  does  pass  the 
lack-of-fit  test.  Its  R-Squared,  adjusted  R-Squared  and  predicted  R-Squared  are  also 
high.  These  results  are  displayed  in  Table  23.  The  coefficients  for  these  two  functions 
appear  in  Appendix  D. 

Table  23.  Statistics  for  the  two  regression  equations 

_ yi  In  U2 

R-Squared  0.9882  0.9920 
Adj  R-Squared  0.9773  0.9847 
Pred  R-Squared  0.9631  0.9686 


5.3  Deterministic  optimum  setting 

The  Hooke-Jeeves  (HJ)  algorithm  finds  the  maximum  value  and  corresponding  op¬ 
erating  solution.  One  thousand  random  starting  points  within  the  experimental  region 
are  generated  using  the  uniform  psuedo-random  number  generator  in  Microsoft  Excel 
Visual  Basic  for  Applications  (VBA).  This  procedure  produces  657  local  maxima. 

The  set  of  local  maxima  is  reduced  using  a  K  means  clustering  algorithm  with 
Matlab’s  kmeans  function.  This  algorithm  classifies  the  657  points  into  K  groups  by 
minimizing  the  sum  of  squares  of  Euclidean  distances  between  the  points  and  their 
corresponding  cluster  centroid  [30]. 

To  find  the  proper  number  of  clusters  K,  K  is  chosen  to  vary  between  2  and  50 
and  the  total  sum  of  squared  distances  in  each  case  is  graphed  against  K  to  find  a 
point  where  the  trade-off  between  the  sum  and  K  is  balanced.  Figure  11  displays  this 
graph. 

Based  on  this  chart,  13  clusters  are  chosen  which  gives  a  total  sum  of  squared 
distances  of  788.05.  The  maximum  value  point  from  each  cluster  is  chosen  to  represent 
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k 


Figure  11.  Total  Sum  of  Squared  Distances  in  all  K  clusters  by  K 

its  cluster  as  a  starting  point  for  the  Monte  Carlo  simulation.  These  points  are 
displayed  in  Table  24. 
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5.4  Monte  Carlo  Simulation 


The  residuals  from  the  regression  equations  are  used  to  estimate  the  marginal 
distribution  functions  for  the  two  responses.  Appendix  D  contains  the  residuals  for 
this  regression  model.  The  lower  bound  A  for  each  response’s  Beta  distribution  is 
chosen  by  rounding  its  lowest  residual  down  to  the  next  integer  value.  The  upper 
bound  B  for  each  response  is  chosen  by  rounding  its  highest  residual  up  to  the  next 
integer  value.  Using  these  bounds,  the  a  and  f3  parameters  are  estimated  using 
Matlab’s  fminsearch  function.  Table  25  contains  the  estimated  parameters  of  the  two 
marginal  Beta  distribution  functions. 

Table  25.  Estimated  parameters  for  the  marginal  Beta  distribution  functions 

Residual  Residual 
Vi  In  y2 

a  17.7591  255.1359 

P  7.9396  255.7977 

A  -4.422E+09  -3 

B  6.389E+09  6 

The  correlation  matrix  R  is  calculated  from  the  residuals  as  shown  in  Table  26. 

Table  26.  Correlation  matrix  of  the  residuals 

1  -0.1912 
-0.1912  1 

Using  Matlab’s  mvnrnd  function  and  the  normal  multivariate  copula  method,  a 
set  of  10,000  sample  residuals  are  constructed  for  the  Monte  Carlo  simulation.  To 
confirm  this  random  sample  is  from  the  intended  distribution,  the  Beta  distribution 
parameter  estimates  and  correlation  matrix  of  the  sample  is  calculated  for  comparison 
with  the  residual  data  estimates.  Table  27  contains  the  Beta  parameters  fit  to  the 
sample. 
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Table  27.  Beta  distribution  parameters  fit  to  the  Monte  Carlo  samples 

Residual  Residual 
Vi  In  y2 

a  18.0711  260.9499 

/3  8.0698  261.6797 

Table  28  contains  the  correlation  matrix  of  the  random  sample.  Both  the  parame¬ 
ters  from  the  two  marginal  distributions  and  the  correlation  matrix  from  the  random 
sample  appear  to  resemble  those  calculated  from  the  residuals. 

Table  28.  Correlation  matrix  for  the  Monte  Carlo  samples 

1  -0.1833 
-0.1833  1 

The  Monte  Carlo  simulation  is  run  using  the  random  sample.  In  this  model,  the 
regression  equation  for  y2  contains  a  natural  log  transformation  on  the  response.  The 
random  sample  is  not  simply  added  to  the  responses.  The  Monte  Carlo  simulation 
allows  one  to  compute  the  expected  utility  for  a  particular  vector  x.  Any  point,  x* 
gives  an  expected  utility, 


^  (lUv(V(h (x*)  +  eu,el^x*^)t  (5.10) 

1=1  '  ' 

where  fi(x*)  is  the  expected  response  yl  given  an  input  of  x*  (i  =  1,2),  n  is  the 

number  of  random  samples  used  in  the  Monte  Carlo  simulation,  eu  and  e2 *  are  the  i-th 

random  samples  from  the  distributions  of  the  residuals  for  y\  and  y2  respectively,  V  is 

the  multiattribute  value  function,  and  Uy  is  the  utility  function  assessed  over  value. 

The  optimization  problem  is  to  then  choose  a  point  a;*  that  maximizes  expected 

utility. 

The  HJ  algorithm  is  started  at  each  local  maxima  displayed  in  Table  24.  Risk 
preferences  are  modeled  by  varying  7  in  the  appropriate  utility  function  in  Equation 
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5.4.  Table  29  contains  the  levels  of  7  chosen  and  the  corresponding  risk  preference 
with  respect  to  y\  and  y2- 


Table  29.  Levels  of  7  sampled  in  Monte  Carlo  simulation 


7 

1U 
iy  1 

1U 

'W2 

-0.01 

-5.751E-12 

-1.015E-06 

0 

0 

0 

0.01 

5.751E-12 

1.015E-06 

0.02 

1.150E-11 

2.030E-06 

Table  30  displays  the  robust  solutions  found  when  7  =  —0.01,0,0.01,0.02  along¬ 
side  the  deterministic  solution.  The  robust  solutions  when  7  =  0,  0.01,  0.02  are  equal. 
They  differ  slightly  from  the  deterministic  solution  and  the  robust  solution  when  risk 
seeking  behavior  is  modeled  (7  =  —0.01). 

Table  30.  Compare  deterministic  solution  with  robust  solutions  for  7  =  —0.01,0,0.01,0.02 


Deterministic 

Solution 

Robust  Solutions 

7 

0.0 1  0,0.01,0.02 

Xi 

0.9 

0.9 

0.9 

x2 

100 

100 

100 

X3 

0.05 

0.05 

0.05 

X4 

0.0675 

0.0675 

0.0675 

x5 

165481 

165473 

165473 

X6 

0.47811 

0.47814 

0.47806 

x7 

0.13 

0.13 

0.13 

X8 

100 

100 

100 

Xg 

4188.76 

4188.84 

4188.84 

X10 

0 

0 

0 

Xll 

0.14081 

0.14081 

0.14081 

X12 

52500 

52500 

52500 

Xl3 

87 

87 

87 

X14 

0 

0 

0 

E(V) 

15.8259 

15.8259 

15.8259 

This  case  illustrates  a  limitation  in  this  method  of  finding  a  robust  optimum 
solution.  The  methodology  assumes  constant  variance  in  the  response  noise  over 
the  experimental  region.  The  Monte  Carlo  simulation  inputs  noise  to  the  responses, 
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yi, . . .  ,yn,  and  calculates  a  robust  solution  by  examining  how  this  noise  affects  the 
multiattribute  value  function.  To  illustrate  this,  consider  the  example  robust  solution 
in  Chapter  2,  redisplayed  in  Figure  12. 


Figure  12.  Illustration  of  Robust  Solution  [16] 

Here,  the  horizontal  axis  represents  the  responses  (y i, . . . ,  yn )  and  the  vertical  axis 
represents  V(yi, ,  yn ).  By  properly  specifying  the  decision  maker’s  risk  preference 
in  the  utility  function  Uv(V(yi, . . .  ,yn)),  the  proper  robust  solution  can  be  found. 

This  method  is  dependent  on  the  form  of  the  value  function  in  use.  The  value 
function  illustrated  in  Figure  12  has  curvature  present.  The  value  function  used  in 
this  situation  is  an  additive  value  function.  It  exhibits  no  curvature.  Therefore  this 
method  will  choose  a  robust  solution  at  or  near  the  optimum. 

These  three  points  are  input  into  NOEM  with  10  replicates  run  for  each  point  to 
test  the  prediction  of  these  optimum  solutions.  Table  31  contains  the  expected  values 
and  95%  confidence  intervals  for  these  three  points. 
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Table  31.  Average  V  and  95%  confidence  intervals  from  10  replicates  in  NOEM  for  the 
three  optimum  points 

Robust  Solution  7  = 

Deterministic  —0.01  0,0.01,0.02 

E(V)  0.6577  0.7156  0.1472 

95%  Conf  Int  [0.5837,  0.7316]  [0.6073,  0.8238]  [-0.1963,  0.4907] 

The  expected  values  ( E(V ))  displayed  in  Table  30  do  not  fall  within  the  95% 
confidence  intervals  found  by  running  these  points  in  NOEM.  This  is  most  likely  due 
to  no  design  point  from  the  experiment  lies  near  these  calculated  optima.  A  space¬ 
filling  design  such  as  a  Latin  hypercube  design  may  present  better  results  due  to  the 
experiment  design  points  being  more  uniformly  spread  throughout  the  experimen¬ 
tal  region.  Augmenting  this  D-optimal  design  with  additional  runs  may  also  have 
presented  a  better  model  of  the  response  surface.  Moreover,  a  Kriging  model  of  the 
system  would  most  likely  have  described  the  surface  better  than  a  regression  model. 
A  Kriging  model  makes  no  assumption  about  the  form  of  the  underlying  system  re¬ 
sponse.  Moreover,  applications  of  Kriging  models  exist  that  model  either  variance 
homogeneity  or  variance  heterogeneity  [26]. 
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VI.  Summary,  Conclusions  and  Recommendations 


Decisions  considering  a  set  of  multiple  objectives  occur  everyday  in  government, 
military,  and  industrial  settings.  After  understanding  the  decision  maker’s  preferences 
regarding  the  trade-offs  between  the  objectives  and  risk  attitude,  a  utility  function 
can  be  constructed  to  provide  insight  to  the  decision  maker  regarding  such  a  decision. 

When  the  decision  situation  concerns  a  system  containing  multiple  inputs  and 
outputs,  response  surface  methodology  provides  a  means  to  model  the  system  with 
a  set  of  equations.  This  set  of  equations  can  be  used  to  inform  the  determination  of 
value  and  utility  functions  that  best  describe  the  decision  situation. 

Many  systems  contain  noise  that  effect  the  system  outputs.  In  these  cases,  the 
best  solution  may  not  be  simply  the  solution  with  the  optimum  expected  output. 
The  best  solution  may  be  in  an  operating  location  where  the  output  varies  less  in  the 
presence  of  noise.  Depending  on  the  form  of  the  value  function,  this  robust  solution 
can  be  modeled  and  found  by  properly  describing  the  risk  attitude  of  the  decision 
maker  in  a  Monte  Carlo  simulation. 

The  desirability  function  has  emerged  as  a  popular  method  of  scalarizing  a  multiple 
response  optimization  problem.  It  is  a  powerful  function  when  the  parameters  are 
chosen  properly  to  describe  the  decision  maker’s  preferences. 

In  the  case  of  the  maximum  is  better  desirability  function,  the  lower  bounds  and 
target  levels  of  the  responses  must  be  carefully  chosen.  A  lower  bound  that  is  too  high 
may  eliminate  desirable  solutions.  A  target  level  that  is  too  low  may  value  multiple 
responses  equally  when  one  is  more  desirable. 

When  the  desirability  function  is  treated  as  a  value  function,  the  deterministic 
trade-offs  described  between  responses  are  dependent  upon  the  responses  and  their 
corresponding  exponents  and  lower  bounds.  Trade-offs  are  not  constant  as  Kros  and 


Mastrangelo  calculated  [29].  Once  the  lower  bounds  and  targets  are  set,  the  exponents 
can  be  adjusted  to  properly  describe  the  decision  maker’s  trade-off  preferences. 

When  a  utility  function  is  assessed  over  desirability,  the  risk  preferences  described 
depend  on  the  form  of  the  utility  function.  In  the  case  of  an  exponential  utility 
function,  the  risk  preference  with  respect  to  the  responses  depends  on  the  responses 
and  their  respective  lower  bounds,  target  levels,  and  desirability  function  exponents. 
Adjusting  the  lower  bound  or  exponent  has  a  non-monotonic  effect  on  risk  preference 
and  must  be  adjusted  carefully. 

When  the  desirability  function  is  treated  as  a  utility  function,  there  exists  an  im¬ 
plicit  assumption  of  utility  independence  between  all  responses.  The  risk  preference 
with  respect  to  each  response  depends  on  the  response,  its  exponent,  and  lower  bound. 
The  desirability  function  can  describe  risk  averse,  risk  neutral,  and  risk  seeking  be¬ 
havior  depending  on  whether  the  exponent  is  less  than,  equal  to,  or  greater  than  the 
number  of  responses.  This  is  in  contrast  to  Kros  and  Mastrangelo  stating  the  risk 
preference  described  by  the  desirability  function  depends  on  whether  the  exponent  is 
less  than,  equal  to,  or  greater  than  one  [29]. 

A  decision  analysis  method  of  calculating  a  robust  optimum  solution  using  a  utility 
function  assigned  over  a  value  function  in  a  Monte  Carlo  simulation  is  applied  to  a 
wire-bonding  process  experiment  from  the  quality  and  reliability  engineering  design 
literature  and  to  a  national  policy-making  scenario  within  NOEM. 

The  usefulness  of  this  method  depends  on  the  form  of  the  value  function.  Since 
random  noise  is  assumed  to  be  constant  throughout  the  experimental  region,  certain 
value  functions  (e.g.,  an  additive  value  function)  are  affected  uniformly  by  this  random 
noise.  When  the  value  function  exhibits  curvature  (e.g.,  the  desirability  function), 
the  effect  noise  has  on  the  variance  in  the  value  function  can  be  measured  and  a 
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robust  optimum  solution  can  be  found  that  is  affected  by  changing  the  risk  preference 
described  by  the  utility  function. 

Additional  research  in  this  area  includes  analyzing  other  multiple  response  opti¬ 
mization  functions.  Any  multiple  response  optimization  function  makes  implicit  and 
explicit  assumptions  regarding  the  decision  maker’s  preferences.  These  assumptions 
should  be  properly  analyzed  so  that  the  decision  maker  can  make  informed  decisions. 

The  methodology  presented  for  finding  a  robust  optimum  solution  presented  in 
this  thesis  provides  insight  in  only  certain  forms  of  the  value  function.  It  does  not 
provide  additional  insight  about  robustness  when  an  additive  value  function  is  used. 
It  can  provide  insight  into  a  situation  when  a  desirability  function  is  used  as  the  value 
function.  Applying  this  method  across  a  larger  set  of  value  functions  should  provide 
a  fuller  understanding  about  the  types  of  functions  for  which  this  method  provides 
insight  about  robustness. 

The  methodology  presented  here  assumes  random  noise  with  constant  variance 
throughout  the  experimental  region.  Further  research  can  include  modeling  hetero¬ 
geneous  variance  and  applying  a  dynamic  random  sample  that  changes  relative  to 
the  variance  modeled  at  the  current  point  in  the  experimental  region  within  a  Monte 
Carlo  simulation. 

The  Kriging  method  can  also  estimate  the  response  surface  assuming  either  ho¬ 
mogeneous  or  heterogeneous  variance  within  the  system.  Further  research  includes 
how  best  to  construct  a  Monte  Carlo  simulation  that  models  the  variance  assumed  in 
a  Kriging  model  of  the  system. 

In  the  current  culture  of  constrained  budgets,  decision  makers  are  faced  more  than 
ever  with  making  decisions  that  encompass  multiple,  competing  objectives.  A  method 
that  properly  describes  the  decision  maker’s  preferences  regarding  risk  and  the  trade¬ 
offs  between  the  objectives  at  hand  while  also  taking  uncertainty  into  account  will 
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serve  to  provide  useful  insight  into  the  decision  at  hand.  With  that  insight,  the 
decision  maker  can  proceed  with  confidence  that  he  or  she  has  made  the  best  decision 
given  current  information. 
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Appendix  A.  Desirability  function  analysis  in  n-dimensions 


Consider  the  desirability  function  in  n-dimensions. 


D  —  (di  ■  ■  ■  dn)n  (A.l) 

Consider  a  system  with  m  inputs,  (xi, . . .  ,xm).  Each  input,  xt  falls  within  the 
experimental  region  defined  by  Xi  G  [x°,  x*],  i  —  1, . . . ,  m.  These  inputs  cause  changes 
in  two  responses,  y\  and  1/2  which  each  fall  within  the  range,  yt  e  [y® ,y*],  i  =  1, . . . ,  n. 
This  analysis  focuses  on  the  case  where  a  maximum  response  is  desired  as  displayed 
in  Equation  2.21  with  no  loss  of  generality  since  the  other  forms  of  the  desirability 
function  can  be  transformed  into  this  case.  Only  the  non-trivial  piece  of  Equation  2.21 
where  L*  <  y.t  <  Tj,  i  =  1, . . . ,  n  is  considered.  Equation  A. 2  displays  this  non-trivial 
piece. 


di  — 


Tj 


1  d/i  <  Vi  <  Ti 


(A.2) 


Where  [Lj,  Tt]  C  [y?,  y* ] ,  i  —  1, . . . ,  n.  This  forces  0  <  D  <  1. 
The  partial  derivative  of  D  with  respect  to  (1  <  %  <  n)  is 
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The  tradeoff  function  between  and  y.j  ( i  ^  j)  is  defined  as 
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The  deterministic  tradeoffs  between  the  two  attributes  can  then  be  stated. 


t{yii  Uj ) 

r3 

Consider  assessing  an  exponential  utility 

U  =  1  -  e~^D  (A. 6) 

The  risk  aversion  of  the  utility  function  with  respect  to  yt  (1  <  %  <  n)  is 

<  =  7^+7^.  (A.T) 

The  risk  aversion  for  the  utility  function  with  respect  to  the  desirability  function 
is  simply 


Vi 


.  Vi  Li 
function  over  D. 


(A.5) 
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(A.8) 


The  contribution  of  the  desirability  function  to  the  risk  aversion  of  the  attribute 


Vi  is 
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The  risk  aversion  expressed  by  this  utility  function  with  respect  to  yt  is 
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Appendix  B.  Data  from  del  Castillo  example 


Table  32.  Data  table  from  wire  bonding  experiment  [9] 


n 

Xl 

Flow 

Rate 

x2 

Flow 

Temp 

x3 

Block 

Temp 

Vi 

Max 
Temp  A 

V2 

Begin 
Bond  A 

Vs 

Finish 
Bond  A 

V4 

Max 
Temp  B 

2/5 

Begin 
Bond  B 

y  e 

Finish 

Bond  B 

1 

40 

200 

250 

139 

103 

110 

110 

113 

126 

2 

120 

200 

250 

140 

125 

126 

117 

114 

131 

3 

40 

450 

250 

184 

151 

133 

147 

140 

147 

4 

120 

450 

250 

210 

176 

169 

199 

169 

171 

5 

40 

325 

150 

182 

130 

122 

134 

118 

115 

6 

120 

325 

150 

170 

130 

122 

134 

118 

115 

7 

40 

325 

350 

175 

151 

153 

143 

146 

164 

8 

120 

325 

350 

180 

152 

154 

152 

150 

171 

9 

80 

200 

150 

132 

108 

103 

111 

101 

101 

10 

80 

450 

150 

206 

143 

138 

176 

141 

135 

11 

80 

200 

350 

183 

141 

157 

131 

139 

160 

12 

80 

450 

350 

181 

180 

184 

192 

175 

190 

13 

80 

325 

250 

172 

135 

133 

155 

138 

145 

14 

80 

325 

250 

190 

149 

145 

161 

141 

149 

15 

80 

325 

250 

180 

141 

139 

158 

140 

148 
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Table  33.  Residuals  from  del  Castillo  example  experiment 


n 

Residual 

Vi 

Residual 

V2 

Residual 

Vs 

Residual 

Z/4 

Residual 

Z/5 

Residual 

y& 

1 

-12.558 

-10.375 

-6.575 

-4.125 

-3.5 

-1.125 

2 

-11.558 

-0.375 

-3.825 

8.375 

3 

4.375 

3 

-14.308 

-5.625 

-15.575 

-5.875 

-2 

-1.875 

4 

11.692 

7.375 

7.175 

6.625 

4.5 

3.625 

5 

10.692 

9.125 

9.8 

8.375 

5.25 

4.375 

6 

-1.308 

-2.875 

-3.45 

-8.625 

-3.25 

-4.625 

7 

-3.558 

1.875 

0.05 

1.625 

0.25 

-1.375 

8 

1.442 

-9.125 

-12.2 

-6.375 

-4.25 

-3.375 

9 

3.067 

2.75 

0.175 

-5.357 

-2.036 

-2.857 

10 

-7.683 

-5.5 

3.175 

-1.607 

-1.536 

-0.107 

11 

8.817 

7.5 

13.425 

-1.107 

2.964 

1.393 

12 

-1.933 

3.25 

8.425 

-1.357 

-0.536 

0.143 

13 

-2.933 

-6 

-6.2 

0.143 

-1.286 

-1.857 

14 

15.067 

8 

5.8 

6.143 

1.714 

2.143 

15 

5.067 

0 

-0.2 

3.143 

0.714 

1.143 
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Appendix  C.  Feasible  Region  of  del  Castillo  experiment 
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Figure  14.  Value  of  D  as  Block  Temp  and  Flow  Temp  vary  (Flow  Rate  =  82.19) 


Figure  15.  Value  of  D  as  Flow  Temp  and  Flow  Rate  vary  (Block  Temp  =  320.30) 
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Figure  16.  Value  of  D  as  Flow  Temp  and  Flow  Rate  vary  (Block  Temp  =  330.45) 
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Appendix  D.  Regression  equation  coefficients  and  residuals 


for  NOEM  experiment 


Table  34.  Coefficients  for  predicted  Debt  (y-[ ) 


Coefficient 


Factor 

Estimate 

Intercept 

-1.2E+11 

A-Adjud  Proc  Time 

-1E+10 

B- Infra  Spend 

-9.5E+07 

C-Govt  Employ 

-5.2E+10 

D- Interest  Rate 

-5.2E+11 

E-Police  Goal 

2223638 

E-Tax  Rate 

1.88E+10 

G'-Adjud  Rate 

2.41E+U 

E-Stimulus  Pet 

3.01E+08 

J-Govt  Wage 

34007472 

E-Service  Spend 

52747611 

L-Corrupt  Pet 

5.39E+11 

M-Police  Init 

485203.9 

E-Jail  Term 

1.7E+08 

O-Stimulus 

2870.185 

AB 

-1.4E+07 

AC 

-5.7E+10 

AD 

1.21E+10 

AE 

34766.79 

AF 

-5.8E+09 

Continued  on  next  page. 
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Table  34.  (Debt  model  coefficients  continued) 


Factor 

Coefficient 

Estimate 

AG 

-4.4E+10 

AH 

-3.5E+07 

AJ 

292440.8 

AK 

30246284 

AL 

1.04E+09 

AM 

9606.421 

AN 

-2E+07 

AO 

276.1704 

BC 

-5.2E+08 

BD 

-7.7E+08 

BE 

300.0216 

BF 

35535699 

BG 

-6.3E+07 

BH 

-205478 

BJ 

-12612 

BK 

-124684 

BL 

-1.3E+08 

BM 

277.5306 

BN 

91005.42 

BO 

-1.04642 

CD 

-1.2E+12 

CE 

578785.7 

Continued  on  next  page. 
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Table  34.  (Debt  model  coefficients  continued) 


Factor 

Coefficient 

Estimate 

CF 

1.22E+11 

CG 

6.9E+10 

CH 

-5.7E+08 

CJ 

20315991 

CK 

-9.3E+08 

CL 

-1.1E+12 

CM 

894714.9 

CN 

9.66E+08 

CO 

14794.99 

DE 

-581941 

DF 

3.29E+10 

DC 

-7.8E+11 

DH 

-8E+08 

DJ 

15254317 

DK 

-7.7E+08 

DL 

-3.3E+11 

DM 

520396.9 

DN 

1.44E+08 

DO 

5747.516 

EF 

-61603 

EG 

-286008 

EH 

-255.663 

Continued  on  next  page. 
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Table  34.  (Debt  model  coefficients  continued) 


Factor 

Coefficient 

Estimate 

EJ 

9.591571 

EK 

267.4753 

EL 

-74454 

EM 

-0.08278 

EN 

-231.426 

EO 

0.003944 

FG 

-5.3E+10 

FH 

-3.5E+07 

FJ 

-495916 

FK 

1.17E+08 

FL 

-7.5E+10 

FM 

10285.56 

FN 

22431989 

FO 

1334.28 

GH 

1.35E+08 

GJ 

-7219620 

GK 

-1.8E+08 

GL 

7.95E+10 

GM 

176310 

GN 

-5222336 

GO 

2426.442 

HJ 

-14446.1 

Continued  on  next  page. 
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Table  34.  (Debt  model  coefficients  continued) 


Factor 

Coefficient 

Estimate 

HI< 

-177820 

HL 

-6.3E+08 

HM 

287.7633 

HN 

-365415 

HO 

13.19157 

JK 

2268.709 

JL 

-6658941 

JM 

-2.26307 

JN 

-1152.77 

JO 

0.126152 

KL 

-2E+08 

KM 

190.0535 

KN 

65779.41 

KO 

-5.14335 

LM 

-129731 

LN 

-6E+07 

LO 

-2756.72 

MN 

285.4206 

MO 

-0.00252 

NO 

3.376809 

A2 

3.2E+10 

B 2 

3978377 

Continued  on  next  page. 
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Table  34.  (Debt  model  coefficients  continued) 


Coefficient 


Factor 

Estimate 

C2 

2.2E+11 

D2 

2.02E+13 

E2 

-16.731 

F 2 

-1.4E+11 

G2 

-2.5E+12 

H2 

-285702 

J 2 

-12046.9 

K2 

-828766 

L2 

-5.4E+12 

M 2 

-5.3217 

N 2 

-4390076 

O2 

-0.00124 

A3 

-2.6E+10 

B3 

-27719.5 

C3 

3.87E+12 

D3 

-1.6E+14 

E3 

4.06E-05 

F3 

2.04E+11 

G3 

1.2E+13 

H3 

-6288.01 

J3 

1.29533 

K3 

6608.289 

Continued  on  next  page. 
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Table  34.  (Debt  model  coefficients  continued) 


Coefficient 

Factor 

Estimate 

L 3 

1.95E+13 

M3 

1.51E-05 

N3 

26475.31 

O3 

8.38E-11 

Table  35.  Coefficients  for  predicted  activists  (1 

Coefficient 

Factor  Estimate 

Intercept 

39.655 

A-Adjud  Proc  Time 

1.762 

E-Infra  Spend 

0.054 

6'- Govt  Employ 

-141.256 

D-  Interest  Rate 

-97.080 

E-Police  Goal 

-0.001 

E-Tax  Rate 

3.105 

G'-Adjud  Rate 

-73.038 

E-Stimulus  Pet 

-0.020 

J-Govt  Wage 

-0.002 

A'-Service  Spend 

-0.009 

A-Corrnpt  Pet 

320.777 

M-Police  Init 

9.50E-06 

Continued  on  next  page. 
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Table  35.  (Activist  model  coefficients  continued) 


Factor 

Coefficient 

Estimate 

At- Jail  Term 

-0.091 

O-Stimulus 

-6.10E-07 

AB 

-0.002 

AC 

12.913 

AD 

-1.894 

AE 

-7.71E-06 

AF 

0.151 

AG 

-4.845 

AH 

0.002 

AJ 

8.75E-05 

AK 

0.007 

AL 

1.518 

AM 

5.64E-06 

AN 

0.003 

AO 

-2.59E-08 

BC 

0.039 

BD 

0.035 

BE 

7.29E-09 

BF 

-0.005 

BG 

0.029 

BH 

-3.30E-06 

BJ 

1.39E-07 

Continued  on  next  page. 


Table  35.  (Activist  model  coefficients  continued) 


Factor 

Coefficient 

Estimate 

BK 

1.39E-06 

BL 

0.013 

BM 

-4.98E-08 

BN 

-6.05E-06 

BO 

-1.71E-10 

CD 

-168.524 

CE 

-1.44E-04 

CF 

-16.719 

CG 

46.315 

CE 

0.065 

CJ 

-0.002 

CK 

0.106 

CL 

192.671 

CM 

6.72E-05 

CN 

0.054 

CO 

-8.55E-07 

DE 

9.77E-05 

DF 

-34.473 

DC 

-81.507 

DH 

-0.030 

DJ 

0.008 

DK 

-0.067 

Continued  on  next  page. 
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Table  35.  (Activist  model  coefficients  continued) 


Factor 

Coefficient 

Estimate 

DL 

-60.734 

DM 

-4.83E-05 

DN 

-0.001 

DO 

-9.20E-07 

EF 

1.98E-05 

EG 

2.05E-05 

EH 

-5.40E-08 

EJ 

-7.05E-11 

EK 

-5.09E-08 

EL 

5.38E-05 

EM 

7.64E-11 

EN 

-6.25E-08 

EO 

5.96E-13 

FG 

-0.475 

FH 

0.006 

FJ 

-0.001 

FK 

0.002 

FL 

-0.111 

FM 

7.26E-06 

FN 

-0.008 

FO 

-6.60E-09 

GH 

-0.026 

Continued  on  next  page. 


Table  35.  (Activist  model  coefficients  continued) 


Factor 

Coefficient 

Estimate 

GJ 

-0.001 

GK 

0.013 

GL 

0.190 

GM 

2.37E-05 

GN 

0.024 

GO 

4.19E-08 

HJ 

-1.26E-06 

HK 

1.38E-05 

HL 

0.046 

HM 

4.55E-08 

HN 

5.80E-05 

HO 

5.04E-10 

JK 

-1.34E-06 

JL 

-3.17E-04 

JM 

6.59E-10 

JN 

9.41E-07 

JO 

8.55E-13 

KL 

0.076 

KM 

-4.62E-08 

KN 

6.84E-05 

KO 

1.08E-10 

LM 

5.68E-05 

Continued  on  next  page. 


Table  35.  (Activist  model  coefficients  continued) 


Factor 

Coefficient 

Estimate 

LN 

0.058 

LO 

7.64E-07 

MN 

-1.84E-08 

MO 

-8.70E-13 

NO 

7.54E-10 

A2 

-7.008 

B2 

-0.001 

C 2 

7228.526 

D2 

2707.111 

E 2 

4.72E-09 

F 2 

-10.416 

G2 

1169.703 

H 2 

8.40E-05 

J 2 

7.16E-07 

K2 

5.42E-05 

L2 

-2686.560 

M 2 

-1.95E-10 

N2 

0.001 

0 2 

1.84E-13 

A3 

5.351 

B3 

6.73E-06 

C3 

-94855.894 

Continued  on  next  page. 


90 


Table  35.  (Activist  model  coefficients  continued) 


Coefficient 


Factor 

Estimate 

D3 

-21722.951 

E3 

-9.94E-15 

F3 

11.252 

G3 

-4959.153 

H3 

5.56E-07 

J3 

-7.44E-11 

K3 

-2.09E-07 

L3 

7234.906 

M3 

1.87E-16 

N3 

-5.68E-06 

O3 

-1.70E-20 

Table  36.  Residuals  from  NOEM  experiment 


Residual  Residual 


n 

Vi 

In  y2 

1 

6.05E+08 

0.01470 

2 

46501523 

0.04140 

3 

-1.4E+08 

-2.26903 

4 

-5.2E+08 

0.02669 

5 

17163091 

0.00104 

6 

2.93E+08 

0.00221 

Continued  on  next  page. 
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Table  36.  (Residuals  continued) 


Residual  Residual 


n 

yi 

In  y2 

7 

6.07E+08 

-0.03120 

8 

-2.1E+08 

0.01059 

9 

-6.3E+07 

-0.00359 

10 

-5.9E+08 

-0.00723 

11 

-8.6E+08 

-0.03596 

12 

5.91E+08 

0.12739 

13 

3.27E+08 

-0.00834 

14 

-4.7E+08 

0.05884 

15 

-8.1E+08 

0.04631 

16 

78070523 

-0.00677 

17 

4.12E+08 

0.02850 

18 

1.1E+08 

0.00031 

19 

6.28E+08 

-0.09888 

20 

3.9E+08 

-0.03841 

21 

7.14E+08 

-0.22464 

22 

-3.1E+08 

-0.11321 

23 

-3E+08 

0.01299 

24 

3.59E+08 

-0.02980 

25 

-5.1E+08 

0.05644 

26 

-8.7E+08 

0.06156 

27 

5.54E+08 

0.14357 

28 

-1.7E+08 

0.01179 

Continued  on  next  page. 
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Table  36.  (Residuals  continued) 


Residual  Residual 


n 

yi 

In  y2 

29 

1.57E+08 

0.00506 

30 

-6.1E+08 

0.19792 

31 

60858431 

-0.07986 

32 

-6.9E+08 

-0.09111 

33 

4.65E+08 

-0.16477 

34 

2.89E+08 

-0.00784 

35 

2.45E+08 

-0.02627 

36 

1.06E+08 

0.00367 

37 

4.94E+08 

-0.03780 

38 

-1.6E+07 

-0.09847 

39 

-2.6E+08 

0.11083 

40 

-1.9E+08 

0.16481 

41 

-8.9E+07 

-0.01404 

42 

-3.4E+08 

0.02320 

43 

-5.5E+08 

-0.03663 

44 

-2.5E+08 

0.09646 

45 

14818895 

-0.05011 

46 

-2.2E+08 

-0.02335 

47 

8.54E+08 

0.09151 

48 

1.62E+08 

-0.00864 

49 

-1.4E+08 

0.00599 

50 

1.95E+08 

-0.03013 

Continued  on  next  page. 
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Table  36.  (Residuals  continued) 


Residual  Residual 


n 

yi 

In  y2 

51 

-6E+08 

-0.06399 

52 

7.77E+08 

-0.06574 

53 

-3.6E+08 

0.00710 

54 

-7.7E+08 

0.05698 

55 

-68764.7 

0.00169 

56 

-6.6E+08 

0.04321 

57 

1.07E+08 

-0.03385 

58 

-2.7E+08 

0.01978 

59 

1.89E+09 

-0.14355 

60 

-4.5E+08 

0.03494 

61 

-5.3E+08 

0.00734 

62 

2.85E+08 

-0.04153 

63 

6.09E+08 

-0.03008 

64 

-4.2E+08 

0.01458 

65 

1E+09 

-0.14547 

66 

-4.4E+09 

0.23652 

67 

-3.9E+08 

0.02111 

68 

1.42E+08 

-0.04746 

69 

7.52E+08 

-0.01527 

70 

4.48E+08 

-0.14286 

71 

5.77E+08 

-0.07107 

72 

2424187 

0.00200 

Continued  on  next  page. 


94 


Table  36.  (Residuals  continued) 


Residual  Residual 


n 

yi 

In  y2 

73 

-3.2E+07 

0.00180 

74 

2.4E+08 

-0.01881 

75 

-2.6E+07 

-0.01237 

76 

1.91E+08 

0.24609 

77 

1.26E+09 

-0.07518 

78 

-5E+08 

0.04178 

79 

-1.3E+08 

0.05549 

80 

7.76E+08 

-0.03767 

81 

2.86E+08 

0.08801 

82 

8.21E+08 

-0.07016 

83 

-2.2E+08 

-0.00163 

84 

3.11E+08 

-0.04712 

85 

2.03E+08 

-0.00505 

86 

-3.6E+08 

0.01480 

87 

4.66E+08 

-0.09495 

88 

-3.8E+08 

0.09892 

89 

5.65E+08 

-0.12924 

90 

-2.3E+08 

0.01413 

91 

-7.2E+07 

0.00131 

92 

1.1E+09 

-0.06148 

93 

-7.7E+08 

0.04685 

94 

1.87E+08 

-0.01776 

Continued  on  next  page. 
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Table  36.  (Residuals  continued) 


Residual  Residual 


n 

Vi 

In  y2 

95 

24635434 

0.00560 

96 

-5.5E+08 

0.03593 

97 

-2.3E+08 

0.02691 

98 

-4.1E+08 

0.01791 

99 

-5.2E+08 

0.03291 

100 

1.09E+09 

-0.06534 

101 

-2.7E+07 

0.17692 

102 

-6.9E+08 

0.13842 

103 

-1.1E+08 

0.10242 

104 

-1.3E+07 

-0.09554 

105 

74725502 

-0.01007 

106 

-9E+08 

0.05072 

107 

-8.6E+07 

0.00804 

108 

7.68E+08 

-0.04576 

109 

77373775 

-0.00131 

110 

-6.2E+08 

0.08858 

111 

4.61E+08 

-0.02004 

112 

2.87E+08 

-0.03326 

113 

5.75E+08 

-0.23979 

114 

-4.3E+08 

0.01492 

115 

20782539 

0.09571 

116 

20906907 

-0.07693 

Continued  on  next  page. 
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Table  36.  (Residuals  continued) 


Residual  Residual 


n 

Vi 

In  y2 

117 

6.44E+08 

-0.02522 

118 

3.28E+08 

-0.01519 

119 

1.39E+08 

0.11427 

120 

-2.3E+08 

-0.06090 

121 

1.02E+09 

0.04245 

122 

-7E+08 

0.14105 

123 

-1.6E+08 

-0.00421 

124 

21789422 

0.00127 

125 

-6.1E+08 

0.03328 

126 

-2.2E+08 

-0.00190 

127 

88528846 

-0.00964 

128 

82073456 

-0.02887 

129 

-5.4E+08 

0.00532 

130 

-7.4E+08 

0.04756 

131 

88697026 

-0.01250 

132 

-7.8E+08 

0.07467 

133 

-9.4E+07 

-0.18327 

134 

-5.2E+08 

-0.00225 

135 

5.25E+08 

0.00848 

136 

3.65E+08 

0.00283 

137 

3.14E+08 

-0.04048 

138 

6.33E+08 

-0.04005 

Continued  on  next  page. 
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Table  36.  (Residuals  continued) 


Residual  Residual 


n 

Vi 

In  y2 

139 

2.58E+08 

-0.00268 

140 

6.45E+08 

-0.09468 

141 

4.89E+08 

-0.06711 

142 

-3.1E+08 

2.29606 

143 

-5.2E+08 

0.02387 

144 

11918414 

-0.00055 

145 

2.85E+08 

-0.04482 

146 

6.06E+08 

-0.03489 

147 

-2E+08 

0.01573 

148 

-7.4E+07 

0.01549 

149 

-5.9E+08 

0.07362 

150 

63301594 

0.08141 

151 

8.05E+08 

-0.21154 

152 

3.28E+08 

-0.00924 

153 

-5.3E+08 

-0.02777 

154 

-8.1E+08 

0.04623 

155 

84505897 

-0.00937 

156 

4.07E+08 

-0.07991 

157 

1.06E+08 

-0.01242 

158 

6.27E+08 

0.04279 

159 

3.99E+08 

0.01167 

160 

6.79E+08 

0.15372 

Continued  on  next  page. 


Table  36.  (Residuals  continued) 


Residual  Residual 


n 

Vi 

In  y2 

161 

-3.5E+08 

0.13814 

162 

-3E+08 

0.02363 

163 

3.6E+08 

-0.01006 

164 

-5.2E+08 

0.01254 

165 

-8.8E+08 

0.04320 

166 

5.77E+08 

-0.19703 

167 

-2.2E+08 

0.01890 

168 

1.68E+08 

-0.03330 

169 

-5.9E+08 

-0.12573 

170 

64175139 

0.07030 

171 

-6.9E+08 

0.17001 

172 

4.65E+08 

0.11906 

173 

3.02E+08 

-0.02906 

174 

2.46E+08 

0.00041 

175 

1.09E+08 

-0.01239 

176 

6.07E+08 

-0.02233 

177 

-2.3E+07 

-0.11000 

178 

-2.6E+08 

0.12963 

179 

-1.9E+08 

-0.15507 

180 

-8.9E+07 

0.01462 

181 

-3.5E+08 

0.00679 

182 

-5.5E+08 

0.10022 

Continued  on  next  page. 
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Table  36.  (Residuals  continued) 


Residual  Residual 


n 

yi 

In  y2 

183 

-2.6E+08 

-0.05576 

184 

14569111 

0.05139 

185 

-2.2E+08 

0.04875 

186 

1.03E+09 

-0.20512 

187 

1.98E+08 

-0.01911 

188 

-1.3E+08 

0.00501 

189 

1.86E+08 

0.00298 

190 

-1E+09 

0.16006 

191 

6.57E+08 

-0.02194 

192 

-1.6E+08 

0.01964 

193 

-7.7E+08 

0.03657 

194 

-1561419 

0.00145 

195 

-4.4E+08 

0.02101 

196 

1.2E+08 

0.01994 

197 

-3E+08 

0.01753 

198 

1.97E+09 

-0.13089 

199 

-4.5E+08 

0.01859 

200 

-6E+08 

0.06112 

201 

1.21E+08 

0.03122 

202 

5.93E+08 

-0.02883 

203 

-4.9E+08 

0.04262 

204 

9.95E+08 

0.04702 

Continued  on  next  page. 
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Table  36.  (Residuals  continued) 


Residual  Residual 


n 

Vi 

In  y2 

205 

-4.4E+09 

0.23771 

206 

-3.6E+08 

0.02866 

207 

1.85E+08 

0.04997 

208 

7.1E+08 

-0.06078 

209 

4.4E+08 

0.08851 

210 

5.66E+08 

0.00929 

211 

-1.7E+07 

0.00199 

212 

-7.9E+07 

0.00215 

213 

2.58E+08 

-0.01779 

214 

-3E+07 

-0.00057 

215 

1.86E+08 

-0.27432 

216 

1.28E+09 

-0.07517 

217 

-5.4E+08 

0.02574 

218 

37521805 

-0.05756 

219 

7.48E+08 

-0.03814 

220 

2.94E+08 

-0.10614 

221 

8.27E+08 

-0.02067 

222 

-2.2E+08 

0.02649 

223 

3.09E+08 

-0.00408 

224 

1.92E+08 

-0.00951 

225 

-2.7E+08 

0.01268 

226 

4.97E+08 

0.04459 

Continued  on  next  page. 
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Table  36.  (Residuals  continued) 


Residual  Residual 


n 

Vi 

In  y2 

227 

-3.2E+08 

-0.04864 

228 

59462511 

0.09071 

229 

-6.5E+07 

-0.01050 

230 

-1.4E+08 

0.01054 

231 

1.14E+09 

-0.06252 

232 

-8.5E+08 

0.04684 

233 

1.94E+08 

0.00299 

234 

-1.2E+08 

-0.00682 

235 

-5.5E+08 

0.01613 

236 

-2.8E+08 

-0.00165 

237 

-5.5E+08 

0.04358 

238 

-5.1E+08 

0.03270 

239 

1.09E+09 

-0.06537 

240 

5.74E+08 

-0.20988 

241 

-6.9E+08 

-0.06992 

242 

-5.7E+07 

-0.08715 

243 

-6.1E+08 

0.13442 

244 

1.34E+08 

-0.00347 

245 

-9E+08 

0.05345 

246 

-8.4E+07 

0.00772 

247 

7.62E+08 

-0.04690 

248 

891461.7 

-0.00482 

Continued  on  next  page. 
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Table  36.  (Residuals  continued) 


Residual  Residual 


n 

Vi 

In  y2 

249 

-6.5E+08 

-0.01392 

250 

3.75E+08 

-0.02906 

251 

2.66E+08 

0.00046 

252 

5.86E+08 

0.17459 

253 

-4.3E+08 

0.03501 

254 

20776848 

0.05542 

255 

20998168 

-0.07753 

256 

6.51E+08 

-0.03974 

257 

3.28E+08 

-0.00574 

258 

1.77E+08 

-0.12397 

259 

-2.4E+08 

0.08025 

260 

1.06E+09 

-0.15035 

261 

-6.4E+08 

-0.05990 

262 

-2.5E+08 

0.02545 

263 

16457326 

-0.00119 

264 

-5.5E+08 

0.02066 

265 

-2.3E+08 

0.04429 

266 

-6.4E+07 

-0.00435 

267 

1.44E+08 

0.01031 

268 

-5.4E+08 

0.06502 

269 

-7.4E+08 

0.02541 

270 

82312932 

0.00280 

Continued  on  next  page. 
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Table  36.  (Residuals  continued) 


Residual  Residual 


n 

yi 

In  y2 

271 

-7.8E+08 

0.01035 

272 

-2.1E+08 

0.19584 

273 

-5.2E+08 

0.06252 

274 

5.24E+08 

-0.07190 

275 

3.56E+08 

-0.03884 

276 

3.34E+08 

-0.00970 

277 

6.4E+08 

-0.03549 

278 

2.06E+08 

-0.01601 
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Appendix  E 


Quad  Chart 


A  Decision  Analysis  Perspective  on  Multiple 
Response  Robust  Optimization 


Introduction 

Decisions  in  which  multiple  objectives  must 
be  optimized  simultaneously  occur  frequently  in 
government,  military,  and  industrial  settings. 
The  desirability  function  is  a  tool  available  to 
assist  the  decision  maker  in  a  multiple 
response  optimization  situation.  Correctly 
specifying  the  decision  maker’s  preferences 
with  respect  to  risk  attitude  and  trade-offs 
between  the  objectives  is  essential  to  modeling 
the  decision  situation  properly. 

In  certain  cases,  noise  exists  within  a 
system  that  affects  response  variability.  A 
robust  solution  accounts  for  expected  response 
as  well  as  variance.  Proper  specification  of  the 
decision  maker’s  risk  preference  while 
modeling  this  noise  produces  a  robust  optimal 
solution  consistent  with  those  preferences. 


Research  Objectives 


■  Analyze  desirability  function  from  decision 
analysis  perspective  to  clarify  assumptions 
inherent  to  this  function 

*  Explore  multiple  response  robust  optimization 
problem  through  Monte  Carlo  simulation  and 
modeling  decision  maker’s  risk  preference 
with  utility  assessed  over  multi-attribute  value 
function 


Robust  Optimization  Method 

*  Estimate  system  model  equations 

*  Assess  multi-attribute  value  function 

*  Find  deterministic  local  optima 

*  Run  Monte  Carlo  simulation  estimating 
marginal  distributions  and  correlation  of  the 
random  noise  from  residual  data 
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Deterministic  tradeoff : 
Desirability  treated  as  value  function 


Point  A  is  the  point  of  maximum  value 
although  this  value  varies  greatly  with  small 
changes  in  the  responses.  This  solution 
would  most  likely  be  preferred  by  a  risk 
seeking  decision  maker. 

Point  B  is  exhibits  less  peak  value  and  less 
variance  than  Point  A.  This  solution  would 
most  likely  be  preferred  by  a  risk  averse 
decision  maker. 

Properly  modeling  the  decision  maker's  risk 
preference  can  produce  the  correct  multiple 
response  robust  solution. 


Desirability  treated  as  utility  function 


:.j  [t  a’  i  t 


Multiple  Response 

Maximum-is-better 

Robust  Optimization 

desirability  function 

Future  Research 

*  Analyze  other  multiple  response 
optimization  functions  from  decision 
analysis  perspective 

■Model  heterogeneous  response  variance 
within  the  system  and  apply  a  dynamic 
Monte  Carlo  simulation  to  identify  multiple 
response  robust  solution 

■  Explore  use  of  Kriging  method  to  model 
expected  response  and  heterogeneous 
variance  within  the  system 


Desirability  function  analysis 
results  overview 

*  Choosing  lower  and  upper  bounds  too 
close  together  can  omit  potentially 
desirable  solutions 

•  User-defined  parameters  specify  trade¬ 
off  and  risk  preferences  described 

•  When  treated  as  a  utility  function,  r, 
controls  whether  the  desirability 
function  describes  risk  avers,  neutral,  or 
seeking  behavior  depending  on  its 
relationship  with  the  number  of 
responses 

*  When  an  exponential  utility  function  is 
assessed  over  desirability,  r{  controls 
whether  the  desirability  function 
describes  increasing  or  decreasing  risk 
aversion  depending  on  its  relationship 
with  the  number  of  responses 
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Figure  17.  Quad  Chart 
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